1. Introduction
The COVID-19 pandemic has been present for almost 2 years since March 2020. Despite the vaccination programs evident in different countries, the continuous rise of infected people still prevails. The increasing number of infected people is due to the different variants and mutations that caused the COVID-19 virus to be highly transmittable, mortal, and sometimes undetectable [
1], as seen in the Omicron variant. The Omicron variant has become evident and continued to spread to different countries, which affected all individuals.
The prominent increase has caused burnout among people [
2] especially healthcare professionals [
3,
4]. Thus, the need to monitor and trace individuals is necessary to reduce exposure and contain the COVID-19 virus spread. Siddiqui et al. [
5] justified that people who are knowledgeable will be practicing preventive measures, however, there is only a weak correlation. Thus, the need to mitigate the COVID-19 virus through monitoring and tracing should be explored.
The COVID-19 tracking and monitoring application/method is available in different countries. In Europe, Kahnbach et al. [
6] showed that there is evidence of high functionality, information quality, and esthetics. However, the engagement–orientation quality was relatively weak. In the United Kingdom, Velicia-Martin et al. [
7] showed that perceived ease of use would lead to perceived usefulness and attitude. Moreover, in Germany, Behne et al. [
8] showed that the application should have an agile set-up and have faster updates towards changes. There are other available contact tracing applications worldwide, however, the need to still consider numerous factors were evident to promote and make the application usable among people in different countries [
9]. Despite the availability of kinds of literature regarding tracing applications, there were limited to none regarding “Thai Chana” tracing application from Thailand.
Thai Chana is the main contact tracing application from Thailand [
10]. It is a self-reporting online tool for contact tracing among Thais. Thailand was able to consider strategies such as surveillance, laboratory testing, case management and control, risk communication, preparation of healthcare staff, facilities, and medical supplies [
11]. Thai Chana helped a lot in the different strategies formulated by Thailand [
12]. It has been required among Thais to be utilized when entering a vicinity. It is stated that the government implemented strict compliance to register with the Thai Chana mobile application, applicable to everyone in Thailand, even the foreign visitors. Thai Chana has the capability to gather information such as name, age, addresses, and contact numbers. In addition, it could indicate and transmit information whether someone infected has been in the area. To which, people are guided whether an area is safe, even the need to isolate, and take the test to mitigate the spread of the virus. However, Bangkok, the capital of Thailand is still considered one of the most highly infected cities after China [
12]. Thus, the need to explore Thai Chana is important to promote usage and would mitigate the infection rate in the country [
13].
Research Questions:
Would the integrated Protection Motivation Theory and Technology Acceptance Model holistically measure perceived usability of a health-related application for COVID-19 contact tracing?
Could a machine learning algorithm solely measure and predict factors affecting human behavior, specifically measuring perceived technology usability?
Are the integrated deep learning neural network and random forest classifier enough to highlight the significant factors affecting perceived usability of a technology?
Could the proposed model and methodology be applied and extended to different studies involving human behavior?
To measure the usability of tracing applications such as Thai Chana, frameworks such as Protection Motivation Theory and Technology Acceptance Model could be utilized. Protection Motivation Theory is a fear and coping appraisal theory that is utilized to measure health-related measures [
14]. Technology Acceptance Model on the other hand is utilized for measuring the usage of a product or technology [
15]. Both studies of Ong et al. [
14] and Prasetyo et al. [
15] integrated the respective theories to holistically measure an individual’s intention or usage behavior. Protection Motivation Theory solely measures a person’s perception of vulnerability and severity [
16]. Van Bavel et al. [
17] considered Protection Motivation Theory to measure the minimization of risk and exposure to improve online security behavior. Mousavi et al. [
18] considered Protection Motivation Theory for privacy protection behavior on social networking sites. Their results showed that privacy assurance played a significant role in people’s usage of a system.
Consequently, the Technology Acceptance Model was considered by several studies for actual usage of a system. Tomczyk et al. [
19] integrated health behavior changes and the Technology Acceptance Model to measure the predicted adoption intentions of the German contact tracing application. Their study considered hierarchical regression modeling and showed that there was only a marginal increase in the predictive value. Moreover, Velicia-Martin et al. [
7] considered Technology Acceptance Model regarding the contact tracing application in the United Kingdom. However, their study utilized partial least square–Structural Equation Modeling. According to Fan et al. [
20], Structural Equation Modeling alone cannot measure the most significant factor due to the causal relationship among the framework considered. The farther the independent variables from the dependent variables may cause low to no significance. In addition, Woody [
21] stated how a mediating effect may be present which hinders the importance and significance level of latent variables due to connections present in a framework. Thus, it would be best to consider utilizing machine learning algorithms such as deep learning neural network and random forest classifier to have high accuracy of prediction among factors affecting human behavior [
22].
This study aimed to determine the perceived usability of the COVID-19 contact tracing mobile application in Thailand, Thai Chana. This was achieved through the integration of Technology Acceptance Model and Protection Motivation Theory to measure the perceived usability of Thai Chana as a contact tracing application in Thailand. This was measured using deep learning neural network and random forest classifier to predict factors affecting the perceived usability of Thai Chana. Specifically, factors such as understanding COVID-19, perceived severity, perceived vulnerability, perceived ease of use, perceived usefulness, attitude towards using, intention to use, actual system use, and perceived usability were considered in this study. This is the first study that considered deep learning neural network and random forest classifier for contact tracing applications. The results of this study would be beneficial for contact tracing applications in different countries as a theoretical foundation for new mobile applications for disease control. Lastly, this framework could also be utilized for another application’s usability among different technologies and its overall acceptance worldwide.
With the trend of research focusing on human behavior dealing with integrated multivariate tools and machine learning algorithms [
22,
23,
24,
25,
26,
27,
28], this study highlighted how machine learning algorithm alone can accommodate analysis involving technology usability. Little to no studies were found to deal with studies that considered sole machine learning algorithm tools in analyzing human behavior, specifically perceived usability of technology with integrated theories as the framework. In addition, this study is one of the first studies that provided evidence for analyzing factors affecting technology usability using combined random forest classifier and deep learning neural network. The flow of the paper is as follows: (1) Introduction that covers the literature review, gap, and background of the study, (2) related studies and theoretical framework, (3) methodology, (4) results and validation, (5) discussion, and (6) conclusion.
3. Methodology
3.1. Questionnaire
Presented in
Table 1 are the constructs considered in this study. Different kinds of literature were considered for the adaptation of the different items utilized. Under Understanding of COVID-19 (U), there were 6 items, perceived vulnerability (PV) and perceived ease of use (PEU) has 5, perceived severity (PS) and perceived usefulness (PU) has 7, attitude towards using (A) and intention to use (IU) has 5, actual system use (AU) has 6, and perceived usability (PUS) has 10. The data were available from answering a 5-point Likert Scale.
Prior to the full distribution of the questionnaire, a pilot test was conducted among 150 responses. The responses collected underwent validation using Chronbach’s alpha test. The result presented a value of 0.836, indicating that the questionnaire is valid for full distribution [
52,
53].
3.2. Participants
Through convenience sampling, a total of 800 respondents voluntarily answered the survey for the perceived usability of the “Thai Chana” COVID-19 tracing application. Utilizing an online survey, the questionnaire was distributed through different social media platforms due to the strict COVID-19 lockdown implemented. Before responses were collected, a question regarding their utility of the Thai Chana application was asked. Only those who utilized the Thai Chana mobile application were considered since other respondents (250) answered they utilized the paper documents. Thus, 800 valid data were considered. From the collected data, a total of 37,600 datasets were considered (respondents and their responses) to run the deep learning neural network and random forest classifier.
3.3. Machine Learning Algorithm
Presented in
Figure 2 is the methodological flowchart utilized in this study. Data acquisition was carried out through an online survey. Data pre-processing is considered a correlation analysis to determine significant indicators for each latent variable. It was seen that 47 total constructs were collected from the study. Following the study of Kuo and Zulvia [
54], those with values less than 0.20 correlation coefficient and
p-value greater than 0.05 were removed due to insignificance. After which, data aggregation of the 8 latent variables were considered as input parameters for the machine learning algorithm. A min_max scalar normalization technique was utilized for the algorithm [
54]. Running the random forest classifier and deep learning neural network, parameter optimization was conducted. Following the study of Chen et al. [
22], parameters for criterion, splitter, training and testing ratio, and depth were considered in this study. For the neural network section, the activation functions, optimizer, number of nodes, number of epochs, and number of hidden layers were optimized [
55,
56,
57,
58,
59]. The following sections provide a detailed explanation for each algorithm.
Following the parameter optimization is the testing for accuracy using cross validation techniques. Following several studies [
56,
57,
58,
59], a 60% threshold was set for the accepted accuracy rate of the classification model. After the creation of the final classification model, validation was conducted to test the created models. After which, interpretation was conducted. As support for the utility of machine learning algorithms, no computational complexity was seen. Liu et al. [
60] explained how the utilization of complex machine learning algorithms would reduce computation time, complexity, and would have higher accuracy. This study considered Python 3.8 to run all algorithms with SKLEARN and Tensorflow packages. Justus et al. [
61] explored the computational complexity of using highly utilized resources and showed that an increase in computation time would be reduced with the current up-to-date technology. From the packages utilized, the main cost of computational complexity would vary from the features used, input data, model complexity, and feature extraction [
62,
63]. Moreover, the more complex the model is, the more training time [
63]. From this study, it was seen that one combination of random forest classifier spent only 0.146 s while deep learning neural network considered 0.480 s. This presents little to no computational complexity with the utilization of the ACER NITRO 5 with core i5 processor, 8 GB RAM, 1 TB Hard disk, Nvidia GeForce GTX 1050 Graphics, running on a Windows 10 system.
3.4. Random Forest Classifier
The purpose of using the random forest classifier is to create a classification model that considers different features coming from the constructs which represents the unobserved variables utilized in this study. Following the study of Gao et al. [
64], the random forest classifier creates a tree model that considers features that are unified, predicts multi-class dimensions, and presents only significant factors for the classification. The advantage of this algorithm is to present only those significant factors and generate a higher accuracy rate compared to other simple classification models [
22]. With the aim to predict and classify factors affecting perceived usability of a new technology among users, random forest classifier may be used to create a classification model for extension and application of predicting influential factors with the integrated theories of Technology Acceptance Model and Protection Motivation Theory.
For the utilization of random forest classifier, data preprocessing was completed by inspecting missing data. The SPSS 25 indicated no missing data. Following this, data cleaning considering correlation analysis was performed. This study considered a threshold for the correlation value of greater than 0.20 with a p-value less than 0.05. From a total of 47 constructs, 34 were considered significant. To which, the data were aggregated to focus on the different factors that influence the usability of Thai Chana application. The factors of U, PV, PS, PEU, PU, A, IU, and AU served as the input nodes for the deep learning neural network. Moreover, PUS served as the output during training.
Data normalization was done and the random forest classifier was utilized considering different parameters such as the criterion (gini and entropy), splitter (best or random), training and testing ratio (60:40, 70:30, 80:20, 90:10), and tree depth (4, 5, 6, 7). Utilizing SKLEARN Packages in Python 4.5, 100 runs of each combination were considered for a total of 6400 optimization runs. It was seen that gini, best, and a 5-tree depth utilizing an 80:20 training ratio presented the highest average accuracy of 92% with 0.00 standard deviation.
3.5. Deep Learning Neural Network
Deep learning neural network is known to be ‘the best model’ for predicting factors or recognizing patterns due to its ability to assess and calculate several perceptions [
65]. Daube et al. [
65] stated how this algorithm resonates with the human-level performance in terms of real-world classification. Luceri et al. [
66] added how deep learning neural network could best predict human behavior, social interaction, subjective thoughts, and feeling. Utilizing deep learning neural network in this study would support the result presented from random forest classifier. Since random forest classifier only predicts significant factors, using deep learning neural network could predict and classify the most impactful factors affecting perceived usability of technology in using the Thai Chana COVID-19 contact tracing mobile application.
Deep learning neural network preprocessing considered data cleaning using correlation analysis, similar to the set conditions with random forest classifier. After data normalization, different activation functions for the hidden layer (sigmoid, tanh, and swish) and output layer (sigmoid and softmax), together with the optimizer (Adam, SDG, RMSProp) was considered. Moreover, the number of nodes was also included for the 80:20 training and testing ratio. To which, a total of 6300 runs were conducted for the feed-forward deep learning neural network process; 10 runs per combination with 150 epochs [
56]. This was conducted to determine the best parameters for the deep learning neural network model.
5. Discussion
This study utilized deep learning neural network and random forest classifier for predicting factors influencing the perceived usability of Thai Chana COVID-19 tracing application. Factors such as understanding COVID-19 (U), perceived severity (PS), perceived vulnerability (PV), perceived ease of use (PEU), perceived usefulness (PU), attitude towards using (A), intention to use (IU), and actual system use (AU) were considered to measure perceived usability (PUS). Based on the result, a 97.32% average accuracy from deep learning neural network was seen and a 92% average accuracy with a 0.00 standard deviation from random forest classifier. Deep learning neural network showed that U had the highest significant effect, followed by PS, PV, and PEU. Consistent with the result from random forest classifier, U and PV led to very high PUS while PS and PEU led to high PUS.
Understanding COVID-19 was seen to be the most significant factor affecting PUS for both deep learning neural network and random forest classifier. The indicators included Thais’ understanding of the COVID-19 virus, its incubation period, symptoms, protocols, when to get the vaccine, and which hospital to go to. The comparison of Asian counties was considered by Wang et al. [
67]. Their study showed that Thais have an 89.8% positive belief regarding the knowledge about COVID-19 and that they know its effect when they are in contact. Caldwell et al. [
68] explained how the knowledge with the minimum health standard could help in the reduction of COVID-19 transmission. In addition, Ning et al. [
69] explained how the success of healthcare professionals regarding interventions would have a positive effect when people know, accept, and understand the disease. Relating to this study, the more people would understand and know about the COVID-19 virus, the more likely they will have the intention to prevent any contraction.
Second, PS was seen to be significant towards PUS among Thais. The indicators included Thais finding of the seriousness of the disease, affects people’s mental health, prolong the outbreak, and that Thailand is more severe than other ASEAN countries. In relation to the study of Fragkaki et al. [
70], when people have a higher perception of severity and high government satisfaction, the more likely they will exhibit behavior change. Based on the results, PS was seen to be the second-highest significant factor. Thus, it could be instigated that Thais a have higher perception of severity, leading to the significant factor affecting the PUS of contact tracing application for reduction of exposure. Mant et al. [
71] explained that when people have high PS, they would change their behaviors to mitigate COVID-19 transmission. For this study, people will have high PUS when they have high PS of the COVID-19 virus. As a support, this result is also consistent with the findings from Trkman et al. [
72].
Third, PV was seen to be a significant factor affecting PUS. The indicators included Thais’ belief in their vulnerability to COVID-19 personally (self, friends, and family), location, and that their country is more vulnerable compared to other ASEAN countries. With that, people have high levels of PUS for the Thai Chana COVID-19 tracing application since they know it will help mitigate the contraction of COVID-19. Boyraz et al. [
73] explained that PV positively affects people’s worries, traumatic experiences, and stress. To which, the more vulnerable the perception is, the higher the will to mitigate contraction. This is in line with the results from De Coninck et al. [
74] from Belgium. They explained how greater health measures were seen in protecting the population when there is a greater belief in PV. Similar to the results of Ong et al. [
37], when people understand the risk, they would highly consider the PV. This would lead to the mitigation in reducing any health-related risks.
Lastly, PEU had a significant effect on PUS among Thai. They believe that Thai Chana as an application can provide clear and understandable information related to COVID-19, successful usage of the application every time, and that the application is easy to use. When utilizing an application, the PEU is usually considered a significant factor [
15]. Prasetyo et al. [
15] explained how PEU can affect the intention to use a certain application. PEU was also seen to be one of the most significant factors affecting the acceptance and usage of technology [
75,
76].
Interestingly, PU and A were not considered significant factors affecting PUS. Guillon and Kergall [
77] explained how the attitude of a person directly correlates with their belief towards the advantage of quarantine. In addition, Guillon and Kergall [
77] explained how trust in the government and health consequences would lead to a high willingness to utilize contact tracing applications. In relation to this study, as long as there is high PS and PV, people will continue to utilize the Thai Chana contact tracing application despite the PU and A. Thus, the advantage of mitigation could be said to revolve around the risk and severity of contracting the virus rather than PU and A. This also supports why IU and AU were not significant. The reason why contact tracing applications are utilized is because of the advantages when it comes to health-related concerns [
78,
79]. The reason for mitigation upon utilizing the contact tracing application among Thais is for reduction of COVID-19 virus contraction, reduce exposure, and safety in general.
Overall, it could be deduced that when people understand COVID-19, as well as its severity and vulnerability, it would affect their perception of the usability of the Thai Chana COVID-19 tracing application. It could be generalized that when there is risk and health-related concerns, people would understand the benefit of utilizing contact tracing applications. Moreover, as long as there is perceived ease of use, then people would continuously utilize the application. Therefore, this should be considered to promote the utilization of contact tracing applications, not only in Thailand, but may also be applicable to other contact tracing applications worldwide.
5.1. Theoretical Implication
The utilization of the machine learning algorithm for human behavior was seen to have different advantages. Machine learning algorithms, such as the artificial neural network, is a type of artificial intelligence that mimics how the body sends a signal to the brain through different neurons that create an output [
79]. Deep learning neural network is a type of neural network that has two or more hidden layers that can process the information further and produce higher accuracy [
54]. Moreover, Vasilev et al. [
80] deep learning neural network has a higher power when it comes to calculation. Ais has been utilized during the COVID-19 pandemic to help in screening, tracking, and predicting future events [
80,
81]. In addition, Jamshidnezhad et al. [
82] utilized a machine learning algorithm for the transmission rate for COVID-19 outbreaks in Iran. Thus, it could be inferred that utilizing machine learning algorithms, such as the deep learning neural network or random forest classifier, may contribute to help mitigating and reducing contraction during the COVID-19 pandemic.
With the utilization of deep learning neural network, it was seen that there was a high accuracy of 97.32% in predicting perceived usability on contract tracing applications. Thus, it could be deduced that the accuracy of prediction is highly reliable. In addition, the 92% accuracy of the srandom forest classifier having consistent results with deep learning neural network further justified the findings. Juarez-Orozco et al. [
83] and Chen et al. [
22] explained how random forest classifier can be a powerful predictive machine learning algorithm tool for human behavior with higher accuracy since it determines the optimum tree among other decision trees produced. Combining both the result for neural network and random forest classifier would therefore be beneficial in predicting human behavior [
22].
5.2. Practical Implication
It was seen from the validation that the results presented well-grounded output in using machine learning algorithms to assess and predict factors affecting human behavior, specifically perceived usability of a technology. Academically, the methodology may be applied and extended in related fields of studies, such as assessing behavior and predicting factors applying machine learning algorithms for classification. It could be deduced that integrating multivariate tools with machine learning algorithms or solely using a machine learning algorithm may be utilized to holistically measure and predict human behavior. Recognizing patterns may be evaluated with the consideration of constructs, and measure items to assess unobserved variables.
Applying the findings of this study, the government may consider implying the severity and vulnerability of COVID-19 in Thailand, comparing it to other countries, and help people understand the COVID-19 virus [
84]. This would promote how advantageous Thai Chana as a contact tracing application is. In addition, the usability of Thai Chana was seen to be easy among Thais. This means that there may be only a few changes needed to be made to promote usage. Based on the findings, it could be seen that the highlight on severity and vulnerability may be capitalized on to enhance the perception of usability, increase motivation, and enhance the Thai Chana COVID-19 contact tracing mobile application’s applicability every day. People already understand the implication, side-effects when infected, negative effects, and health complications when infected with the virus. Thus, there is only a need for highlighting the vulnerability and severity in promoting the mobile application. Moreover, the government may enhance motivation by indicating the benefits, both community-wise and health-wise. In addition, the government may also promote the utilization of Thai Chana by campaigning and addressing the public’s concerns. The findings of the study from Munzert et al. [
43] showed that people questioned the effectiveness of different contact tracing applications when they are not promoted properly. Therefore, the need to highlight the use, its intention, and application would help people in using the Thai Chana COVID-19 contact tracing mobile application.
5.3. Limitations
This study may have attained a high accuracy rate for both machine learning algorithms, however, there are still limitations. First, this study was conducted only in Thailand and focused on the main contact tracing application, Thai Chana. Other contact tracing applications were not considered. Future researchers may consider other contact tracing applications and compare them to determine the significant factors to generalize the findings. In this way, the contact tracing application may be generalized to enable promotion and usage [
82]. Second, this study only considered two machine learning algorithms, deep learning neural network and random forest classifier. Other machine learning algorithm tools may be considered and compared to promote the utility of other algorithms applicable [
65]. Classification tools such as support vector machine, and optimization techniques such as particle swarm optimization may be applied. In addition, the location was not considered. The urban and rural areas may have different results depending on their understanding of COVID-19 [
85,
86,
87], perceived severity, and perceived vulnerability towards utilizing the Thai Chana contact tracing application. Thus, clustering may also be applied to enhance the findings of the study, such as KMeans algorithm or Fuzzy CMeans. Lastly, the majority of the age group considered in this study was between 15 and 24 years old only. Due to the COVID-19 lockdown implemented, an online questionnaire was only utilized. As explained by Ong et al. [
14], most of the generation utilizing social media platforms are at a younger age. Thus, it is recommended to consider in-person data collection to consider broader and wider age groups. Moreover, findings may be different based on the effect of the virus, perception, and even health-related disease outcomes among the different age groups. Moreover, interviews may be conducted to consider the qualitative measures applicable in measuring perception, knowledge, understanding, and usability errors. A qualitative–quantitative approach may then be utilized to highlight and generalize the perceived usability of Thai Chana as a COVID-19 contact tracing mobile application. This may produce results that may be utilized for workshops, and even create possible techniques that are applicable to other studies.
6. Conclusions
The progress in mitigating the COVID-19 contraction has been slow. Due to new variants and the mutation of the COVID-19 virus, the vaccination program was not able to keep up with the progression. To which, countries implemented contact tracing applications to help reduce exposure to the COVID-19 virus [
88,
89,
90,
91]. However, there is a lack of study regarding the different applications available. Specifically, the Thai Chana COVID-19 contact tracing application has been underexplored. Thus, this study aimed to predict factors affecting the perceived usability of Thai Chana in Thailand integrating Protection Motivation Theory, Technology Acceptance Model, and System Usability Scale using deep learning neural network and random forest classifier. Specifically, factors such as understanding COVID-19 (U), perceived severity (PS), perceived vulnerability (PV), perceived ease of use (PEU), perceived usefulness (PU), attitude towards using (A), intention to use (IU), actual system use (AU), and perceived usability (PUS) were considered in this study.
With the 800 voluntary Thais participants in the survey, a total of 37,600 datasets were considered. Applying the deep learning neural network, the results produced an average accuracy of 97.32% and 92%, with 0.00 standard deviation for random forest classifier. From the results, U was seen to be the most significant factor, followed by PV, which leads to high PUS. Moreover, PS and PEU were considered significant with high PUS among Thais in using the Thai Chana contact tracing application. When people understand the COVID-19 virus, their perception of severity, and perceived vulnerability would lead to a high perception of contact tracing usability. This means, in order to promote the utilization, the government should instill awareness of the severity and vulnerability among people of the COVID-19 virus. This will also help people to understand how the virus can affect them, leading to the continuous usage of Thai Chana. With that, the government may capitalize on the finding of this study to promote motivation for continuous usage of the mobile application. When people see how the mobile application may help promote positive health-related behaviors and highlight the effect of reduced vulnerability and severity, an increase in motivation for the utility would be applied. As explained by Siddiqui et al. [
5], people who are knowledgeable will practice promoting mitigation and intention to reduce the negative effects of COVID-19. In addition, Chuenyindee et al. [
13] explained how implementing the utilization of technology does not necessarily promote positive usage. Thus, with the proposed suggestions, people will be highly motivated to use the system.
The consideration of the machine learning algorithm may be highlighted in this study. With high accuracy rates, it was validated that the machine learning algorithms may be utilized for assessing and predicting factors affecting human behavior, not just in assessing technology usability. The disadvantage was the time it took to finish the optimization process for the different parameters to enhance the accuracy of the models considered. In addition, the advantage is the model and parameter setting that was considered, which could be utilized for other studies considering human behavior. In addition, despite several factors considered, the high-quality machine learning algorithms were able to assess the study effectively. Thus, machine learning algorithms could be used to assess perceived usability of technology and other related studies such as natural disasters and even education [
90].
The framework and findings of this study may be considered by other countries for their independent contact tracing application. It was seen that when the citizen understood the impact of the virus, how it could affect the health of the people negatively, and how the contact tracing mobile application would help in mitigating the negative side-effects, then people would be more likely to consider and utilize a system to reduce it. It could be deduced that the integrated framework holistically measured health-related technology, its application, and usability among user. Moreover, this may be considered to promote the utilization of contact tracing applications, not only in Thailand [
91], but may also be applicable to other contact tracing applications worldwide. Lastly, future researchers may consider the method and framework utilized in this study for the evaluation of applications and systems worldwide.