Design of an Educational Chatbot Using Artificial Intelligence in Radiotherapy

Abstract

Context: In cancer centres and hospitals particularly during the pandemic, there was a great demand for information, which could hardly be handled by the limited manpower available. This necessitated the development of an educational chatbot to disseminate topics in radiotherapy customized for various user groups, such as patients and their families, the general public and radiation staff. Objective: In response to the clinical demands, the objective of this work is to explore how to design a chatbot for educational purposes in radiotherapy using artificial intelligence. Methods: The chatbot is designed using the dialogue tree and layered structure, incorporated with artificial intelligence features such as natural language processing (NLP). This chatbot can be created in most platforms such as the IBM Watson Assistant and deposited in a website or various social media. Results: Based on the question-and-answer approach, the chatbot can provide humanlike communication to users requesting information on radiotherapy. At times, the user, often worried, may not be able to pinpoint the question exactly. Thus, the chatbot will be user friendly and reassuring, offering a list of questions for the user to select. The NLP system helps the chatbot to predict the intent of the user so as to provide the most accurate and precise response to him or her. It is found that the preferred educational features in a chatbot are functional features such as mathematical operations, which should be updated and modified routinely to provide new contents and features. Conclusions: It is concluded that an educational chatbot can be created using artificial intelligence to provide information transfer to users with different backgrounds in radiotherapy. In addition, testing and evaluating the performance of the chatbot is important, in response to user’s feedback to further upgrade and fine-tune the chatbot.

1. Introduction

User interfaces in computer applications can be in different forms such as graphical user interfaces, command lines, menu driven, form based and natural language [1,2]. Recent development of computer technology, Internet of Things (IoT) and artificial intelligence (AI) generated more advanced interfaces such as chatbots and virtual reality [3,4,5]. A chatbot is a software application designed to replace humans to provide user communication through an on-line chat conversation. Our research team proposes that there is a need to make the chatbot humanlike, by simulating the ways a human would behave and respond in the user conversation, especially when covering emotional and sensitive topics [6,7,8]. The chatbot is talking to real human beings, often worried and in distress. A chatbot should not just be vocalizing the written content available on a website. It should be constructed like a fictional character in literature, to whom the user can feel connected, who can be a reassuring voice in times of need. We named our chatbot the RT Bot.

Technically, the RT Bot has the necessary features, complete with routine testing, turning and updates necessary to pass the standard of the Turing test [9,10]. Nowadays, a chatbot in any domain is very popular as a virtual assistant to answer questions from users in business, industry and healthcare [11,12,13].

First, the RT Bot must be loaded with the necessary information. It is not just reading out information regarding cancer health that can be found in cancer centre or hospital websites. Such websites usually provide a list of topics which are neither interactive nor offer any personal touches, in attempting to understand the concerns of the users. Cancer patients, families and practitioners, on the other hand, are in challenging, often stressful, situations, wanting to access accurate information as efficiently as possible. On top of this, there is seldom any information specifically on radiotherapy, despite the large number of cancer patients, especially those with advanced stage (metastatic) disease that have to go through this process. Having “someone” with professional knowledge who can “listen” to them, provide the medical information with good will and encouragement goes a long way to help patients and families struggling with death [14,15]. As it is quite impossible to have real medical professionals to stand by around the clock to answer questions especially during the pandemic period, an educational chatbot would be essential to disseminate health information to the users. For example, the radiation treatment process is often a black box for cancer patients and families. They may scramble for answers everywhere, from visits to libraries, to Google search, asking Siri, or posting their questions to WhatsApp groups. However, they may not be getting the sort of accurate information they need. Recent innovative computer technology and technique such as IoT and AI apps in mobile devices can be used to provide the information about radiotherapy needed by patients and their families [16,17]. However, these still need to be customized to be relevant to the user’s query.

The performance of a chatbot can be enhanced by AI, which makes the chatbot more powerful than a conventional website and Google searches, because the machine learns from the user’s request and improves its knowledge base with each interaction [11]. It is situational, providing answers specific to the situations and environment of the query [18]. Machine learning (ML) and natural language processing (NLP) are two branches of AI generating the chatbot. Machine learning can read text to discern the sentiment of the user through natural language understanding [19]. The NLP applications try to understand natural human communication and respond to the user using a similar and natural language [20]. Machine learning is used to understand vast nuances in human language and to learn to answer in a way that the user is likely to comprehend. The innovation in AI through powerful ML algorithms and NLP can enable the chatbot to hold conversations with the user without too much human intervention.

Medical chatbots can carry out different functionalities. In cancer treatment, Bibault et al. [21] have investigated and found that chatbots can create bi-directional information exchange with patients, which could be leveraged for the treatment process, screening and follow-up. This AI-assisted chatbot can be deployed over various modalities such as text messaging, mobile applications and chat rooms. In healthcare, Chung et al. [22] proposed a chatbot-based healthcare service with a knowledgebase for cloud computing. They proposed a mobile health service based on a chatbot in response to accidents or change of conditions of patients with chronic disease, which may occur in everyday life. Lokman et al. [23] designed a chatbot that can function as a virtual diabetes physician. The chatbot allows diabetic patients to have diabetes management advice without going to the hospital. Hajare et al. [24], on the other hand, proposed a chatbot that not only can answer each and every query asked by the end user, but also focuses on a local database as well as a web database for educational purposes. The chatbot is built making use of the most recent technologies such as ML, NLP, pattern matching, data processing algorithms to enhance the performance. Setiaji et al. [25] draws attention to a human-to-machine conversation model using knowledge in a database. This chatbot includes a core and an interface that accesses that core in relational database management systems. The database stores the knowledge, and the interpreter stores the programs of function and procedure sets for pattern-matching requirements. The above AI-assisted chatbots in medicine show that the main challenges are to train the chatbot to understand the context of the user, to learn how the chatbot handles open-ended or unstructured inputs from the user, especially about emotional or sensitive topics.

In this paper, we present the design of an educational chatbot for different kinds of users: cancer patients and their families, general public and radiation staff working in the cancer centre or hospital. We will cover the different stages of design, including defining the problem, creating conversational flow, chatbot training, testing and evaluation, maintaining and updating. We also discuss the programming and implementation issues we encountered when creating the educational chatbot in radiotherapy. The objective of the chatbot is to conduct a basic and comprehensive information transfer from the chatbot provider to the user regarding simple knowledge in radiotherapy, customized for different kinds of users. The chatbot is designed taking advantage of some AI features such as NLP and informed by communication strategies for the targeted audience.

2. Materials and Methods

2.1. Overview of the Workflow

2.1.1. Identification of User Group

First, the scope of the project has to be defined by identifying the types of potential users and their needs. Such information can be gathered through workshops, meetings and conferences. In radiotherapy, the user groups may include the radiation staff working in the cancer centre, cancer patients, and people from the general public including the patient’s family members. After understanding their needs, a chatbot can be created to address the issues. The chatbot is designed to simulate conversation with different kinds of human users through the internet. Figure 1 shows an example of the basic workflow of the chatbot. In Figure 1, the chatbot begins with questions identifying the user. The user may be a radiation staff member such as a radiotherapist or medical physicist looking for specific radiation treatment information in their fields (e.g., radiation safety), or a patient or his/her family member looking for general information about a specific radiation treatment process (e.g., external beam radiotherapy). Both groups of users have their specific needs. For example, radiation staff may want to confirm some specific rules and policies in radiation safety [26], while patients may want to understand some basic terms in external beam radiotherapy [27]. Therefore, it is necessary to classify and direct the user into his or her groups linked to the related database.

The basic conversional flow for the chatbot is shown in Figure 1, when the user asks the chatbot what information about cancer treatment he wants to know. Then the chatbot will start to identify and categorize the user: e.g., a radiotherapist. If the answer is yes, the chatbot will direct the user to the staff user group. If the answer is no, further identification will follow. When the user finds it difficult to answer such a question, the chatbot will provide guidance or a choice. Such an initial communication approach through Q and A also builds the connection between the chatbot and the user. From the responses of the user, the chatbot tries to predict the type of information the user is looking for. This involves creating a set of intents, entities and actions that the chatbot will be able to recognize and respond to. In case the user cannot answer well or the chatbot cannot understand the response from the user, guidance will be provided by the chatbot to help the user to focus. The application of AI can help the chatbot to predict the response (intent) from the user accurately and provide more humanlike conversation.

2.1.2. Identification of Different Scenarios

The chatbot will put on a different persona with respect to the user’s group. It first identifies the user’s needs and then follows a suitable scenario tailored to the user based on the needs. To create an educational chatbot in radiotherapy, we use three user groups in the pilot, namely, patient and family, general public, and radiation staff or student. The approach and response of the chatbot as per each scenario to the user are different. For example, if the user is identified as a patient, the chatbot will act as a friend or mentor with friendly wording for the user. However, if the user is a radiation staff member, the chatbot will act professionally like an assistant trying to provide the requested information accurately, quickly and precisely. The scenarios can be a cancer patient looking for information about brain surgery, a user from the general public looking for information about breast cancer screening, or a medical physicist looking for dosimetric data on radiation beams. The chatbot needs to be trained with the data in the scenario of each user group, including both the input and expected output. Then the chatbot can understand and respond to different users appropriately. These scenarios can be identified by healthcare providers and clinical workers based on their experiences through interviews, meetings or workshops. If the chatbot cannot communicate to the user due to unexpected or unstructured wording, guidance will be provided by the chatbot to help the user. This guidance will be included in the script and workflow of the chatbot. Any unanswered questions will be added to a corpus of questions to be used to train the chatbot, so that it can become smarter with ML. This iterative approach is necessary, using what we learn from the former session to influence the inputs for the subsequent session. Improving the chatbot through iterations has the advantage of delivering the chatbot quickly, and while it is undergoing continuous improvement.

2.1.3. Communication Framework with Dialogue Tree and Layered Approach

The chatbot is designed using two phases of a communication framework. The first phase considers structured questions and answers based on the scenarios for patients and family, the general public and radiation staff. This means proceeding with a set of questions and answers like multiple choices, but with tips and feedback from the chatbot. This minimizes human monitoring and maximizes usage. In the second phase, after identifying the popular scenarios for each of the three user groups, the chatbot is added with the ability to answer open-ended questions asked by the users. This phase allows AI to learn from the Q and A through ML. We need to do this in Phase 2 after identifying the most needed scenarios in Phase 1 and build sufficiently substantial big data to train the chatbot in Phase 2 [26,27].

In the interaction logic of Phase 1, the user can select related items step by step as per his or her needs and finally acquire the information. For example, the chatbot first identifies the user’s background as shown in Figure 2a. When the user selects the user group, a list of items will be provided. The list is based on the scenarios previously identified for the user group. For example, the general public likely wants to know information about cancer statistics, healthy life style and cancer screening, while radiation staff may want to know information about the radiotherapy process, radiation safety and so on. If the user is a patient and selects “Cancer sites”, then a further breakdown of the item will be followed as shown in Figure 2b. The chatbot will ask the user which cancer sites he or she is interested in. A list of cancer sites such as brain, head-and-neck, breast and so on is shown to the user for selection. In this case, if the user selects “Brain”, then further breakdown of such an item will be shown in Figure 2c. In the list, the user can select which category regarding brain cancer he or she wants to know about. If “Brain Radiotherapy” is selected, the chatbot will provide information related to such a topic. This is different from websites as this works through Q and A with the chatbot.

In Phase 1, for example, the interaction logic of the chatbot starts with a brief introduction and asks what it can help the user with (e.g., Figure 3: “Hello! I am RT Bot. How can I help you?”). The chatbot can answer most direct questions regarding radiotherapy such as “What is radiotherapy?” typed in by the user as shown in Figure 3. After answering, the chatbot follows if the user wants to know the definition of radiotherapy or how it works. However, if the chatbot does not understand the user’s text, a systematic guidance will be provided to the user as a structured Q and A set.

2.2. Artificial Intelligence used in Chatbot

The architectural diagram of the chatbot created on the IBM Watson Assistant platform can be seen in Figure 4. The chatbot is created by the Watson Assistant and can be integrated into social media or customized IoTs such as cell phones. The Watson Assistant provides AI features such as NLP and is linked to the IBM Cloud. The Cloud is further connected to other Watson services such as speech to text conversation and the backend systems.

The chatbot is powered by AI, which analyses the user’s response, breaking down the text or speech to work out rules and recognize patterns. This means with the application of ML, the chatbot can convert the user’s response into a structured format. One ML tool used in chatbots is the NLP technology which is included in various chatbot development platforms such as the IBM Watson Assistant [28,29]. The NLP includes pattern matching and linguistic analysis. For example, the IBM Watson Assistant can specifically recognize keywords from the user’s response and weigh them to determine the intent of the sentence. This is then cross-referenced with the database of intents to evaluate the response that the chatbot can provide.

When the chatbot is communicating with the user using the Q and A approach, the questions are integrated individually into the dialogue nodes. While each node is self-contained, it also leads to the next question. The dialogue tool provides the user with the ability to access the dialogue tree. This allows the tree to control the dialogue nodes completely. The dialogue tree of the IBM Watson Assistant is shown in Figure 5. At a given node, such as the “Welcome” node in Figure 5, the user can give a variety of responses. In the Watson Assistant platform, for example, it is possible to set the response variation to either random, sequential, or multiline. To create a general flow of the dialogue tree, each node has an option button allowing the chatbot developer to add a new node either below or above it. Each node also has an option to have child nodes and this is important considering there are various ways that a user can answer a question.

A powerful tool provided by the IBM Watson Assistant to build the chatbot is the Intent, which allows the chatbot to process the user input and predict the intent of the user [30]. This tool is supported by the ML and NLP of the platform. As shown in Figure 5, for certain nodes of conversation to be accessed, the chatbot has to recognize the specific intents in the user’s input. One example is the Hint node, which will pop up when the user does not understand the question from the chatbot and can only be accessed when the user asks for help. The node can be created by the Intent node to evaluate the possible user inputs as a response to that node. For example, if the user types “help” or “hint” as an answer for a question of the chatbot, an intent called “#Hint” is called. This and other intents are created by feeding the chatbot examples of the intent. The chatbot then runs these examples through ML and NLP and learns to recognize them from user input.

It should be noted that some intents require less than a few examples as the chatbot expects the user to answer only “Yes” or “No”, as there are limited ways to write them. However, in some specific cases such as name collection when the chatbot asks the name of the user, the Intent works alongside Entities using the “Annotate Entities” function as shown in Figure 6. When the chatbot collects the user’s name, the only part of the input that the chatbot is interested in is the name itself but it is not interested in other words. Therefore, the function of the “Annotate Entities” is to select the region of interest in the example sentences (i.e., the blue boxes in Figure 6). In this way, the chatbot may categorize any future names as entities, even the ones that it has not seen before. For the tool of Entities, the function can be considered a database that the chatbot has access to, can cross-reference and extract data. For example, a list of 6000 most common names in America can be used as a database [8]. The chatbot can extract names, when recognizing the related “#Collect_name” intent as shown in Figure 6.

3. Results

To demonstrate the communication flow and running at the start, the chatbot created based on the architectural scheme in Figure 4 begins by introducing itself and asking the user questions to categorize the user into patient, radiation staff, from the general public or from the patient’s family as shown in Figure 7.

In this example, the user identified himself as radiation staff in the cancer centre. The chatbot then displayed a list of items showing a variety of information that the user might want to know (Figure 8).

The contents of the list included different categories of information such as patient consultation, simulation, treatment planning, treatment delivery, patient follow-up, quality assurance and so on. This list was designed specifically for the radiation staff. For patients and the general public, they had their own list of information. When the user (radiation staff) selected “Simulation” from the list, the chatbot further asked the user to clarify if he or she was interested in “CT-SIM” or “Conventional SIM”. These are two common treatment simulation methods in radiotherapy. To answer that, the user only needed to click the icons to obtain the information as shown in Figure 9.

The workflow in Figure 7, Figure 8 and Figure 9 can be found in Figure 10 for the group of radiation staff.

Sometimes, the user may ask the chatbot a question directly. For example, the user asked the chatbot “What is clinical trial?” in radiotherapy as shown in Figure 11. In that case, the chatbot answered the question and asked the user if he or she wanted to know more, such as different phases of clinical trial. The user could then select a phase (Phase 1–4) from the chatbot. In Figure 11, the user picked up Phase 1. The chatbot would then explain further the Phase 1 trial in greater details.

4. Discussion

4.1. NLP and ML

George Bernard Shaw said: “Progress is impossible without change, and those who cannot change their minds cannot change anything.” [31]. ML brought about sweeping changes in recent years and accounted for significant progress in medicine such as in creating medical chatbots. The Turing test is used to evaluate the intelligence of the machine, and the test is passed if a human being cannot distinguish the machine from another human being through conversation [32]. Since then, NPL was developed to enhance interactions between computer and human language. In the past, NLP depended on a set of hand-written rules coupled with dictionary look up to learn and understand the language from the user. It can be seen that such hand-written rules would only become more and more complex and unmanageable. ML, on the other hand, can simplify and enhance the learning process because the computer can automatically focus on some common cases selectively based on the ML algorithm. These automatic learning procedures supported by, for example, neural networks, can help to generate models to manipulate unfamiliar and erroneous user’s input.

NLP includes a number of developed tasks in text and speech processing. These tasks include word segmentation, text-to-speech, stemming, sentence breaking, relationship extraction and so on [33]. Compared to other conversational and prescriptive medical chatbots, the goal of our educational chatbot is to provide information transfer to users who may be radiation staff, patients or people from the general public. Therefore, the main focus of the NPL in our design is to understand and classify the expression from user’s with different backgrounds. In creating such a chatbot, some NLP tasks were used such as (1) Name Entity Recognition: this process helps to find out the entity of a person, location and organization from the user’s input; (2) Intent: this process helps to execute an appropriate action to achieve the user’s goal; (3) Context: this process differentiates the user’s input to investigate if the message may have different meaning in the conversation; and (4) Entity Linking: this process helps to link any words which are referred to an entity such as a popular location, a well-known company or a famous person.

4.2. Programming and Implementation

In programming and running the educational chatbot, one issue found was the lack of global ability to restart the conversations freely by the user, as the chatbot is deposited on an IoT. For the IBM Watson Assistant, though the “Clear” function in the developer tools allows the conversation to reset back to the first node, when creating the chatbot in the platform, such a function is not available once the chatbot is published. Therefore, to implement this ability globally, every single node needs to be reworked manually. The solution is to add a new intent, linked to an exit node back to the first node. However, adding an exit node to some key parts of conversation would result in unforeseen error because the initial design of logic in programming did not foresee the need for this functionality.

Another programming issue in creating an educational chatbot for radiotherapy is the lack of ability to perform mathematical operations for the context variables. For example, when the chatbot offers a radiation safety test to the user (radiation staff) as training, it is desired that the chatbot could calculate the final mark of the test and provide a letter grade or percentage result to the user. In this case, the results can only be displayed in text format, where each question is either “Correct” or “Incorrect”. When creating the chatbot, the development tool does not allow the developer to add up the context variables of the result of each question.

For the AI-assisted chatbot, NLP is used to understand and interpret user inputs. Sometimes, this is challenging and can be a difficult task due to the complexity and variability of human language. Moreover, the chatbot may have difficulty in understanding the context of a conversation. To avoid a poor understanding of open-ended or unstructured user inputs, and to address the difficulty in identifying intents and entities, the chatbot will first identify the user’s background and classify the user into a user group (e.g., patient, radiation staff and general public). This can direct the chatbot to access the data related to the user’s background. In addition, to further avoid confusion and inappropriate responses, the chatbot will include guidance with fixed options to help the user.

For integration and implementation, the chatbot can be included in a website. IBM Watson Assistant allows the developer to integrate the chatbot into different third-party media such as WhatsApp with Twilio, Slack, Facebook Messenger, SMS with Twilio and Intercom as shown in Figure 4. The chatbot can also be deposited into a website setting up through, for example, Weebly or WordPress. Since the educational chatbot in radiotherapy is quite specific, depositing it into a website allows the chatbot to be found by web crawlers. This ensured that if someone looks for a radiotherapy chatbot on a search engine, the chatbot can be found readily. This would not be the case if the chatbot was implemented through WhatsApp or Facebook.

For the privacy and security concerns, our educational chatbot only provides a single directional information flow from the chatbot to the user. Therefore, there is no storage and record of any user information such as personal data in the chatbot. Moreover, we do not find any significant time delay issue in the chatbot created using the IBM Watson platform.

4.3. Testing and Updating

Although the chatbot is powered by innovative computer technologies such as ML to provide humanlike communication to the user, various human user tests are necessary to find out any bugs in the chatbot and to evaluate its performance. This involves testing the chatbot with real users or simulated user input, and evaluating its performance based on accuracy and user satisfaction. The tests can be conducted by inviting different stakeholders to use the chatbot, including, people from the general public, patients and radiation staff, to test the chatbot for their respective databases. In the test, an evaluation template can be set up to measure different metrics regarding the user’s experience in using the chatbot. For example, the test can ask the evaluators to score their experience on a scale in a range of grades, based on the metrics of information quality, user experience and navigability, or the test can be designed following the recent international standards for measuring the “quality in use” of medical chatbots [34]. The test can be carried out remotely through email invitation, or in person in a research meeting or conference, gathering a number of potential users such as radiation staff. To further evaluate and improve the chatbot, workshops for healthcare workers working with patients and families can be conducted to introduce the chatbot and ask them for feedback. These workshops are decent and essential in quality assurance to improve and fine-tune the chatbot.

When the educational chatbot is created and implemented, a continuous update of the contents and features are necessary. These include making adjustments to the conversational flow, adding new intents and entities or updating the training data. Although the chatbot has been validated before implementation, it may find it difficult to handle some edge cases which may not be covered by the training data, or the user may use an unexpected input. Moreover, some small errors which are missed in the validation process due to human variation may exist. These errors would be found occasionally by the users. Their feedbacks to the developer are important to further improve the chatbot. This process would take time but is worthwhile to continue in order to maintain the quality of the chatbot. Moreover, the content of the chatbot needs to be updated routinely to keep pace with new developments in radiotherapy, such as new cancer treatment techniques, new radiation safety policies or new statistical data in radiotherapy. The chatbot should be subject to continuous improvement, updated and modified frequently based on the feedback from the user, new features (e.g., new NLP algorithm) offered to the chatbot and new information in radiotherapy.

4.4. Chatbot Content Management

Building the content for the chatbot is a time-consuming process as the scope of information demanded by different groups of users can be extensive and under continuous update. Each item of information is subject to further breakdown of sub-items and so on. Since each item in the category is unique, the chatbot should provide information in a simple and comprehensive way as per different user classes. A careful selection and filtration of the content is necessary. For example, if the user asks for information about “Brain radiotherapy”, a simple explanation should be displayed with a further question following a customized chatbot conversation stream. In this case, the chatbot can further ask the user if he or she wanted to know more about different options of brain radiotherapy such as gamma knife, cyber knife or stereotactic radiosurgery offered by a medical linear accelerator. In addition, plain English should be used for patient and user from the general public while technical terms should be used if the user group is radiation staff.

5. Conclusions

A chatbot integrated into the IoT can be designed using a layered structure and dialogue tree approach for educational purposes in radiotherapy. By taking advantage of some AI features such as NLP offered by some development platforms like the IBM Watson Assistant, the educational chatbot can be created with humanlike character. Using the Q and A approach, the chatbot can communicate with the user from different backgrounds and provide guidance to the user who is difficult to acquire information. It is concluded that the chatbot is not a one-off finished product, but needs to be updated and improved continuously to respond to the user’s needs, and to keep pace with the advancements of computer technology and radiotherapy. Further work will include adding functional features in the chatbot and enabling the chatbot to handle multiple languages rather than English only.