Artificial Intelligence-Driven Oncology Clinical Decision Support System for Multidisciplinary Teams

Watson for Oncology (WfO) is a clinical decision support system driven by artificial intelligence. In Korea, WfO is used by multidisciplinary teams (MDTs) caring for cancer patients. This study aimed to investigate the effect of WfO use on hospital satisfaction and perception among patients cared for by MDTs. This was a descriptive study that used a written survey to gather information from cancer patients at a hospital in Korea. The rate of positive change in patient perception after treatment was 86.8% in the MDT-WfO group and 71.2% in the MDT group. In terms of easily understandable explanations, the MDT-WfO (9.53 points) group reported higher satisfaction than the MDT group (9.24 points). Younger patients in the MDT-WfO group showed high levels of satisfaction and reliability of treatment. When WfO was used, the probability of positive change in patient perception of the hospital was 2.53 times higher than when WfO was not used. With a one-point increase in overall satisfaction, the probability of positive change in patient perception of the hospital increased 1.97 times. Therefore, if WfO is used appropriately in the medical field, it may enhance patient satisfaction and change patient perception positively.


Introduction
Treatment options for cancer patients are becoming more diverse, and treatment guidelines are changing more rapidly; similarly, medical staff need to possess more comprehensive medical information. Medical staff thus spend much time updating their knowledge alongside direct patient care [1]. Oncologists work 56.7 h per week, of which 4.6 h on average are spent updating and maintaining their knowledge of their field [2]. Additionally, as the treatment methods for tumor patients become more diverse and complex, objective evidence and opinions of medical professionals in various fields are necessary to determine the most appropriate treatment methods. Multidisciplinary teams (MDTs), wherein medical professionals from various fields gather to determine the treatment methods for individual tumor patients, have been used in earnest since the 1980s to improve the quality of patient treatment [3]. Although MDTs have the advantage of positively influencing treatment methods for patients, enhancing patient satisfaction, and decreasing the burden of the medical staff, it has been reported that they are inadequate in applying updated medical findings [4][5][6]. Clinical decision support systems (CDSSs) have been developed to solve this problem. CDSSs are artificial intelligence (AI) computer systems used by medical staff to improve healthcare quality by increasing the pace and accuracy [7]. It can identify crucial information from the medical records of patients, find relevant evidence, and suggest appropriate diagnoses and treatment methods. A CDSS has at least two core components: the knowledge base and inference engine. It allows medical staff to retrieve relevant Therefore, in this study, we assessed patient satisfaction with and perception of a hospital, based on specific experiences and investigated how WfO affects these variables. We intended to analyze patients' experiences with an AI oncology CDSS and suggest future courses of action.

Study Design and Data Collection
This was a preliminary descriptive study investigating the effects of WfO use on MDT-treated patients' satisfaction with and perception of the treating hospital. The participants were cancer patients who received treatment from MDTs at G Hospital from March to September 2018. G Hospital is private university hospital. This study was approved by the Clinical Research Ethics Committee of G University (GDIRB2018-109). We explained the purposes and methods of this study to the patients and conducted a written survey among 293 participants who voluntarily agreed to the survey. The final analysis included data from 285 people; eight individuals were excluded due to missing data.

Patient Satisfaction and Perception
We investigated patient satisfaction and perception using structured questionnaires. Questions were created based on literature reviews and information collected from a preliminary meeting with the MDTs. Questions included in the survey covered the following points: general characteristics of the patient, the use of WfO, the difference between the general treatment and WfO, change in the patient's perception of the hospital after treatment, patient's satisfaction, and the reliability of treatment methods. General characteristics of patients included sex, age, whether he/she was a new patient, disease stage, and type of cancer. Participants were classified as patients with MDTs using WfO (MDT-WfO) or patients with MDTs that did not use WfO (MDT). MDT-WfO or MDT groups were determined according to the doctor's preference, since hospitals guarantee the doctor's choice of medical treatment. Differences between groups were investigated through a yes-or-no question asking whether the treatment provided by an MDT differed from traditional treatment, which is a single doctor dedicated to patient care. We analyzed the satisfaction score relative to the clinical characteristics of the patient, since these factors may have been correlated with one another. Changes in patients' perception of the hospital after treatment were classified as a positive change, negative change, or no change. Patient satisfaction consisted of overall satisfaction with treatment and satisfaction with specific domains-including medical facilities, clarity of explanations from the medical staff, attentive listening by the medical staff, and sufficient treatment time. The reliability of treatment methods was measured by the level of the patient's confidence in the final treatment methods. Satisfaction and reliability were measured on a scale of 1 to 10 points, with higher points indicating higher satisfaction and reliability.

Data Analysis
We analyzed patients' satisfaction with and change in perception of the hospital relative to patient characteristics and whether or not WfO was used. General characteristics were presented as frequencies and percentages. The χ 2 and Fisher's exact tests measured the change in the patients' perception of the hospital, as well as the difference between general medical treatment and treatment from an MDT relative to two factors: general patient characteristics and whether WfO was used or not. Patient satisfaction and the reliability of treatment methods were analyzed using a t-test, ANOVA, and Kruskal-Wallis test. A logistic regression analysis was performed to identify the factors affecting change in patient perception to bring about either a positive change (1) or no change (0). In the regression analysis, only variables with p < 0.05 or less were examined using univariate analysis. The Hosmer-Lemeshow test was used to test the goodness of fit for logistic regression models. SPSS for Windows version 23.0 (IBM Corp., Armonk, NY, USA) was used for all statistical processing.

General Characteristics of Patients
Out of the 285 patients who participated in this study, 129 (45.3%) received treatment from an MDT using Watson for Oncology (MDT-WfO) and 156 (54.7%) received treatment from a general MDT. The sample included 253 (88.8%) new patients who had never received treatment from an MDT before. There were 125 men (43.9%) and 160 women (56.1%). The average age of the participants was 60.6 ± 12.9 years. Among the participants, 69 (24.2%) participants were in disease stage 1, 54 (18.9%) in stage 2, 84 (29.5%) in stage 3, and 78 (27.4%) were in stage 4. The most common cancer type was colorectal cancer (n = 108; 37.9%), followed by gynecological cancer (n = 55; 19.3%), breast cancer (n = 48; 16.8%), lung cancer (n = 36; 12.6%), liver cancer (n = 14; 4.9%), gastric cancer (n = 13; 4.6%), and thyroid cancer (n = 11; 3.9%). There were no differences in sex and age ratios between MDT-WfO patients and MDT patients. However, there were differences in terms of patient visit (first visit or revisit), disease stage, and cancer type. The MDT-WfO group had many first visit patients. The most common disease stage was stage 3, while the least common was stage 4. Conversely, among those in the MDT groups, stage 4 was the most common, while stage 2 was the least common. In the MDT-WfO group, colorectal cancer was the most common, followed by breast cancer. Colorectal cancer was also the most common in the general MDT group, followed by gynecological cancer; breast cancer was the least common (Table 1).

Patient Perception
Two hundred and twenty participants (77.2%) reported that the treatments they received (MDT-WfO or MDT) were different from treatment from a general single doctor. There was no significant difference in the responses according to the type of MDT or characteristics of the participants.
Change in patient perception of the hospital after treatment was categorized as a positive change, negative change, or no change. None of the participants reported negative changes, while 223 (78.2%) reported positive changes and 62 (21.8%) reported no change. The change in perception according to the type of MDT was statistically significant (χ 2 = 10.183; p = 0.001). Out of 129 MDT-WfO participants, 112 (86.8%) answered that their perception of the hospital changed positively post treatment; meanwhile, 111 out of 156 (71.2%) participants in the MDT group reported positive changes in perception. Relative to sex, significantly more women than men reported that they had a positive change in perception of the hospital after treatment (χ 2 = 5.102; p = 0.024). As for cancer type, the rate of positive change in perception of the hospital was the highest among breast cancer patients and was found to be statistically significant (χ 2 = 15.254; p = 0.018) ( Table 2).

Patient Satisfaction and Reliability
Out of 10 points, the average overall satisfaction with treatment was 9.42 ± 1.02 points. Satisfaction was the highest in terms of attentive listening by medical staff (9.49 ± 1.02 points) and sufficient treatment time (9.49 ± 1.01 points). The average reliability of the treatment methods was 9.44 ± 1.02 out of 10. When analyzed according to the treatment group, it was found that the MDT-WfO group had higher overall satisfaction than the MDT group. In particular, satisfaction with easily understandable explanations was significantly higher (t = 2.136; p = 0.034) in the MDT-WfO group (9.53 ± 0.98 points) than in the MDT group (Table 3).

Patient Satisfaction and Reliability within the Treatment Group
Compared to those 60 years old and older, patients 59 years old and younger in the MDT-WfO group were found to have significantly higher satisfaction with medical staff listening and sufficient treatment time and higher overall satisfaction, as well as a significantly higher reliability of treatment methods (Table 4). In the MDT group, the reported satisfaction and reliability of patients 60 years old and older were higher than the group 59 years old and younger; this difference was not significant (Figure 1).

Patient Satisfaction and Reliability within the Treatment Group
Compared to those 60 years old and older, patients 59 years old and younger in the MDT-WfO group were found to have significantly higher satisfaction with medical staff listening and sufficient treatment time and higher overall satisfaction, as well as a significantly higher reliability of treatment methods (Table 4). In the MDT group, the reported satisfaction and reliability of patients 60 years old and older were higher than the group 59 years old and younger; this difference was not significant (Figure 1).

Effect of Patient Satisfaction and WfO on Change in Patient Perception
The univariate analysis determined that WfO use, sex, patient satisfaction, and the reliability of treatment significantly affected patient perception. In the multivariate model, in which all eight variables were used as independent variables, overall satisfaction and the use of WfO led to positive changes in patient perception of the hospital (Table 5). When using WfO, the probability of positive change in perception of the hospital was 2.53 times higher (95% CI 1.29-4.97; p = 0.007). Finally, when the overall satisfaction increased by a point, the probability of positive change in patient perception of the hospital increased 1.97 times (95% CI 1.17-3.32; p = 0.011).

Discussion
Evaluating the satisfaction and experiences of patients is vital in understanding and enhancing the quality of medical care [18]. Technology has improved the efficiency and quality of healthcare services; this is exemplified by the use of systems such as CDSSs and other AI devices to enhance medical staff performance and patient experience [7]. A patient's satisfaction and experience with their treatment is not only a crucial indicator of the quality of care received by the patient but also influences a patient's adherence to treatment and thereby affects treatment outcome [19]. Favorable patient satisfaction and experiences increase the likelihood of the patient being faithful to treatment, resulting in positive health outcomes; thus, patient satisfaction and experience can be an indicator for the indirect measurement of the quality of medical care. Patient satisfaction and experience are also related to the degree to which the medical staff meet patient expectations. Therefore, medical staff must consider how to determine and execute priorities to efficiently fulfill patient expectations [20]. In this study, we investigated what patients expected from MDTs using WfO by analyzing patient perception and patient satisfaction. We suggest that patient expectations be treated as foundations for future studies focused on patient-centered care. The main findings of this study can be divided into the following three sections.
First, the analysis of the patients' perceptions of the hospital after treatment showed that the change in perception was significantly more positive in patients in the MDT-WfO group than in patients in the general MDT group. Therefore, it can be said that the use of WfO has a positive effect on patient satisfaction and perception of the hospital. Analyses based on sex and cancer type revealed that patients who were women or had breast cancer had higher rates of positive perception change. Additionally, logistic regression analysis identified factors affecting the changes in patient perception of the hospital. Univariate analysis showed a significant relationship between patient perception and WfO use, sex, patient satisfaction, and reliability of treatment methods. However, after controlling variables, the multivariate analysis found that only WfO use and overall patient satisfaction had significant effects on patient perception. The probability of a positive change in patient perception of the hospital increased 2.3 times when WfO was used. Positive perception can continuously prompt patients to return to the hospital, bring about a positive clinical outcome, and enhance patient safety by increasing the continuity of treatment [21]. In this study, the use of WfO was found to be the most influential factor in changing patient perception, thus proving the value of WfO in improving clinical outcomes.
Second, satisfaction with easily understandable explanations showed statistically significant differences between groups. Using WfO can help medical professionals explain treatment options to their patients. Science is rapidly changing, thus increasing the number of treatment options available; however, medical staff often have difficulty obtaining the latest information. Even experts such as oncologists tend to have knowledge limited to the treatment of specific cancers, thus making it difficult for them to suggest a variety of novel treatments [22]. WfO allows medical professionals to explain treatments to their patients based on up-to-date information. Listening to explanations based on the most recent scientific evidence may improve patients' understanding of their disease and treatments for it. It is of utmost importance for patients to be involved in their care and receive the right amount of information [17], since patient satisfaction increases when sufficient information is received and when psychosocial needs related to treatment are met [19,23]. A study that analyzed data from the MEPS in the United States suggests that poor satisfaction among patients is related to increased use of the emergency department rather than the outpatient clinic [24]. One plausible explanation could be that, when satisfaction with hospitals and medical staff is low, patients often do not come to the hospital and tend to not follow the prescriptions given by their doctor. As such, time may be lost before receiving appropriate treatment. Patients then experience complications of the illness that may necessitate immediate attention only obtainable from an emergency department. Therefore, it is necessary to try to improve patient satisfaction. Given that MDT-WfO leads to higher patient satisfaction and positive perception compared to general MDT, using WfO may facilitate appropriate treatment among patients, which, in turn, may lead to positive health outcomes.
Third, WfO can have a positive impact on younger patients. In a meta-analysis that examined the relationship between patient satisfaction and sociodemographic characteristics, age showed the strongest correlation with satisfaction. Generally, satisfaction among elderly patients has been reported to be high [25]. However, the results of this study found that younger patients from the MDT-WfO group showed high satisfaction and reliability of treatment methods. Thus, WfO can raise compliance with medical treatment in younger patients with low treatment satisfaction. A study wherein participants were divided by age into two groups-younger and older than 55 years old-suggests that the information provided by the medical staff is associated with treatment dissatisfaction in patients under 55 [19]. With the help of WfO, medical staff can provide patients with scientific data and sufficient information and explanations; this may improve overall patient satisfaction and reliability of treatment in younger patients.
In sum, among patients receiving treatment from MDTs, WfO is crucial in improving patient satisfaction and positively changing patient perception of the hospital. Previous WfO-related studies focused mostly on the concordance rates between treatment methods chosen by medical staff and those by WfO and how these concordance rates served as both an indicator of the accuracy of WfO and an essential factor influencing patient outcomes. The accuracy of treatment is medically important and precedes patient satisfaction and perception. Previous studies on the accuracy of WfO indicate that a limitation of WfO is its inability to consider certain characteristics of the individual and their environment and culture; these factors include insurance coverage, medical guidelines, race, and geographical region. Additionally, blind trust in WfO can pose problems for communication and acceptance of treatment between medical staff and patients [13]. As such, the accuracy of WfO is to be determined. However, while accuracy takes precedence, patient satisfaction and patient perception are also crucial factors that directly and indirectly affect patient outcomes by changing patient compliance and attitudes toward care. This study found that the use of WfO has a positive effect on the empirical aspects of patient experience. As such, the proper use of WfO in healthcare can lead to positive results in the future.
One limitation of this preliminary study was that it was a questionnaire survey conducted among patients of one university hospital. To be able to generalize the findings of this study, future studies must measure patient satisfaction in various settings. Additionally, the data collected in the study were voluntarily self-reported; publication or survivor bias and recall bias may have affected the information presented. A second limitation was the limited characteristics studied-namely sex, age, and clinical characteristics-in reference to the reliability results. In future studies, it may be beneficial to consider other factors, such as levels of patient anxiety and differences in delivery between medical teams. Third, this study only investigated patient satisfaction and perception relative to whether WfO was used or not among patients being treated by MDTs; we lacked data on patient satisfaction prior to the study. Finally, patient satisfaction and perception are not defined clearly in the existing literature; hence, there is still a need to examine various facets of patient experience. As such, further studies on the differences before and after the use of WfO, differences in medical staff performance, and various definitions of patient satisfaction in various settings should be analyzed.

Conclusions
The results of this study confirmed that WfO influences patient satisfaction and leads to positive patient perception among patients being treated by MDTs. Specifically, WfO use was found to be the most influential factor that could induce positive change in patient perception of the hospital. WfO use was found to increase patient satisfaction and reliability of treatment methods in younger patients, who generally tend to be less satisfied with their treatment. Patient satisfaction and perception have direct and indirect relationships with the patient's medical results. Therefore, patient outcomes can be improved by employing MDTs with WfO.