Employing a Chatbot for News Dissemination during Crisis: Design, Implementation and Evaluation

Abstract

The use of chatbots in news media platforms, although relatively recent, offers many advantages to journalists and media professionals and, at the same time, facilitates users’ interaction with useful and timely information. This study shows the usability of a news chatbot during a crisis situation, employing the 2020 COVID-19 pandemic as a case study. The basic targets of the research are to design and implement a chatbot in a news media platform with a two-fold aim in regard to evaluation: first, the technical effort of creating a functional and robust news chatbot in a crisis situation both from the AI perspective and interoperability with other platforms, which constitutes the novelty of the approach; and second, users’ perception regarding the appropriation of this news chatbot as an alternative means of accessing existing information during a crisis situation. The chatbot designed was evaluated in terms of effectively fulfilling the social responsibility function of crisis reporting, to deliver timely and accurate information on the COVID-19 pandemic to a wide audience. In this light, this study shows the advantages of implementing chatbots in news platforms during a crisis situation, when the audience’s needs for timely and accurate information rapidly increase.

1. Introduction

A “bot” (a shortening of “robot”) is any automated software that responds to incoming information or data, using predetermined rules and/or artificial intelligence to select a response [1]. Chatbots refer to any software application that engages in a dialog with a human, using natural language. The term is most often used in connection with applications that converse via written language. However, with advances in speech recognition [2], they can be programmed to respond differently to certain keywords or employ machine-learning techniques to adapt their responses based on words included in queries [3,4,5].

For several years, social, conversational bots have been used to provide benefits to companies, who use them to reduce time-to-response and provide enhanced customer service, increasing satisfaction and engagement [6]. Chatbots consist of a specific AI-based software category developed by companies, to automate communications and management of transactions with their customers [7,8]. Use of this technology in news media platforms, although relatively recent, offers many advantages to journalists and media professionals and, at the same time, facilitates users’ interaction with useful and timely information. Initially, when newsbots were interacting with users on social media platforms, such as Twitter and Facebook, they tended to mainly function as re-broadcasters of traditional news content to social media [9], namely to alert, aggregate and monitor content for their users. However, news chatbots have moved a long way forward and are now able to offer a variety of informational services to their users, from the dissemination of information to data categorization and taxonomy.

This study aims to show the usability of a news chatbot during a crisis situation and uses the 2020 pandemic crisis of COVID-19 as a case study. Drawing from Piccolo et al. [10], Ford and Hutchinson [11], and Radziwill and Benton’s (2017) [6] studies regarding the use of chatbots in crisis situations, the basic targets of this research are to design and implement a chatbot in a news media platform with a two-fold aim in regard to evaluation: first, the technical effort of creating a functional and robust news chatbot in a crisis situation, both from the AI perspective and interoperability with other platforms, which constitutes the novelty of this approach; and, second, users’ perception regarding the appropriation of this news chatbot as an alternative means of accessing existing information during a crisis situation. The news chatbot designed was evaluated in terms of effectively fulfilling the social-responsibility function of crisis reporting to deliver timely and accurate information regarding the COVID-19 pandemic to a wide audience. In this light, this study shows the advantages of implementing chatbots in news platforms during a crisis situation, when the audience’s needs for timely and accurate information rapidly increase.

2. Literature Review

2.1. Chatbots in the News Media: Uses and Affordances

In recent years, robotic technologies have established themselves in the news media by offering practical alternatives for journalistic daily routines. Although the transition from bots to chatbots has not been easy, automation techniques have found their way to the news media platforms, eventually resulting in “automated news” and “automated journalism” through the use and exploitation of algorithms.

Lokot and Diakopoulos (2015) [9] argue for the “potentially positive and beneficial utility of automated news and information sharing”, including how bots may “contribute to positive effects in the public media sphere if employed ethically and conscientiously” (p.3) [12]. This new wave of automation incorporates many forms of what is commonly referred to as “artificial intelligence” or “cognitive technologies”, which aid both the inputs and outputs of journalism [13].

Automation initially aimed to reduce human effort and facilitate the work journalists undertake to get news to the public in the era of big data [8]. The application of chatbots in journalism has shown that they can unburden journalists from daily routine work, reducing pressure to produce quantity and allowing them to concentrate instead on quality, freeing up capacity for in-depth analysis and reporting [5]. This also enables them to consider best practice in journalistic work, like checking multiple sources, reflection and diligence [14]. In addition, chatbots embedded in news platforms assist in the categorization and taxonomy of news, especially in the era of big data, facilitating users’ effort to locate specific news topics they are interested in [15].

Moreover, the application of chatbots in the news media reshapes the narrative of journalism: Not only do they allow the personalization of information delivery and immediate interaction among sources and recipients [5], but they also do so through trusting speech, by which they seek to generate emotion and foster loyalty [16] and, consequently, trust.

In the news media environment, trust remains one of the most important features that journalists around the world strive to achieve. Several studies have pointed toward trustworthiness that users ascribe to third-party computerized actors and algorithmic processes used for selecting and curating the news. Thurman et al. (2018) [17], for example, demonstrate a link between respondents’ trust in news organizations and their assessment of the utility of algorithmic news selection. They argue that, as trust in news organizations falls, people are less likely to recognize editorial selection by journalists as a good way to receive news. By contrast, agreement that automated personalization is a good way to receive news is less affected by distrust in the media. They conclude that users do not recognize the link between automated news personalization and the operation of news organizations, “believing the technology has a degree of immunity from contamination by a politically compromised or untrustworthy news media” (p.17). Ford and Hutchinson (2019) [11] frame their study in terms of the newsbot as a medium that mediates existing social relationships (in this case, between audiences and media organizations). Following Hoflich (2013) [18], who suggest that robots can be either a connective or divisive element in the inter-group relationships that they mediate, Ford and Hutchinson find that the friendly newsbot contrasts favorably with users’ previous experience with traditional dissemination of news and the journalists who produce it. Shneiderman (2020) [19] introduces a two-dimensional framework of Human-Centered Artificial Intelligence (HCAI) that separates levels of automation/autonomy from levels of human control. This model seeks to produce computer applications that are reliable, safe and trustworthy (the three RST goals). Achieving these goals, especially for complex poorly understood problems, can support, among others, mastery, creativity and responsibility; as Shneiderman (2020) [19] argues, well-designed automation preserves human control where appropriate, thereby increasing performance and enabling creative improvements (p.495). In addition, the design of “shared-control systems”, where a virtual human model assists the human in a control task—by simultaneously co-controlling, a system could add value for chatbots’ users [20]. It is known that if the assisting virtual human model does not accurately characterize human control, this may lead to increased workload for the human operator, instead of improved performance (Griffiths and Gillespie, 2005).

In recent years, chatbots have been connected to “sensor journalism”, the use of sensors embedded in the real world as a source of data driving the composition and distribution of automated news items. Examples of this “sensor-journalism” or “sensor-telling” have covered topics such as pollution and animal welfare [17,21]. Several studies in this area are mainly evidential in (a) exploring whether and why news consumers think automated personalization is a better way to receive news than selection by journalists and editors [17]; (b) questioning received wisdom on the existence of filter bubbles [22]; and (c) even asking whether recommendation engines might promote, rather than limit, diverse news exposure [23].

However, recent developments regarding the use of news chatbots have brought significant implications for individuals, society and communication, resulting in the formation of Human–Machine Communication (HMC) as an area of research [24,25], as did previous changes in technology that were followed by corresponding shifts in communication research. While the underlying theoretical assumption in the majority of communication research is that it is humans who are communicators and machines that function as mediators, within HMC, this assumption is challenged by asking what happens when a machine steps into this formerly human role. Lewis et al. (2019) [26] argue that HMC has to be re-envisaged as a way of approaching the study of technology based on its function as communicator (message source), rather than merely as mediator (message channel).

Moreover, the way(s) news chatbots are used can raise issues regarding data ownership and privacy. In their study regarding the news chatbot created by the Australian Broadcasting Corporation (ABC), Ford and Hutchinson (2019) [11] argue that the way the ABC operated its news chatbot could have consequences for Public Service Media (PSM) accountability principles, since the ABC was dependent on the private infrastructure of Facebook and ChatFuel. They state that such matters will only become more relevant as news is delivered increasingly via smart speakers and controlled by the few global corporations with the expertise, data and resources necessary for leading such developments.

2.2. Employing News Chatbots for Crisis Reporting: The Case of the COVID-19 Pandemic

Until the early days of 2020, the global societies of the Western world could conceive of a global pandemic scenario only within the framework of a Hollywood movie. The spread of COVID-19 found mankind unprepared to deal with emergency health crisis situations, although, in the previous decade, several countries around the world had confronted lesser health crises, such as the SARS (Severe Acute Respiratory Syndrome) pandemic in the early 2000s. The COVID-19 pandemic made clear to the whole world that diseases do not recognize national borders, whereas national health systems—even in the most developed countries—are unable to support an effective response to such crises.

In this framework, crisis reporting has become more relevant than ever and in need of more robust and effective practices to fulfil its social-responsibility function: accurate, timely and precise information available to all citizens on practical issues in regard to the global pandemic. As Kruger (2005) argues [27], reporting information “as fully as its standing as a major public health crisis demands, and reporting it in all its various aspects” (p.127). However, crisis reporting regarding health issues in the past had to meet these demands in a way that was in accordance with the technological tools available. The technological explosion of Web 3.0 in recent years seems to have weaponized media organizations with the ability to fulfil their social role with a variety of tools. Among these, chatbots offer a practical solution for both audiences and media professionals to deal with a severe global health crisis, provided they are used to fulfil an audience’s specific needs and demands.

In their study regarding the use of chatbots in crisis situations, Piccolo et al. (2018) [10] find that users expect an ideal chatbot to be high performing (fast, efficient and reliable), smart (knowledgeable and accurate in predictions), seamless (easy and smooth) and personable (understands me and is likeable). Their model was based on Radziwill and Benton’s study (2017) [6], which compiles a set of quality attributes expected for chatbots classified into six categories: (1) performance, reflecting the ability to deal with unexpected input and the appropriate escalation to humans; (2) functionality, which is related to linguistic accuracy; (3) humanity, referring to humanized interactions; (4) affect, which encompasses enjoyability, politeness and personality traits; (5) ethics and behavior, referring to respecting users’ privacy, sensitivity to social concerns and trustworthiness; and (6) accessibility, by detecting users’ meaning or intent and responding to social cues.

Dale (2016) [2] states that, as the technology evolves, in the near future, it could become hard to distinguish a chatbot from a human in a conversation. Louwerse et al. (2005) [28] argue that having “humanlike” behavior can be one of the success criteria in the process of evaluating chatbots. However, in the case of news chatbots, this clear distinction is important, so as to avoid frustration. Design plays a significant role, and, in most cases, it is up to the designer to decide the “form” of the chatbot. For example, creating a chatbot by using a “humanlike” form can make the application more “user-friendly”; however, it can also change user perception of it as “human or machine”. Overall, embedding human elements in a news chatbot can be identified as an important step toward building trust in the chatbot application and in the sociotechnical initiative as a whole [10].

Looking for further success criteria to evaluate chatbots, Zamora (2017) [29] goes back to the virtual agents’ literature from the 1990s, which includes being efficient in responsiveness. To this end, an effective chatbot relies not only on the user interface design but also on the development of a robust AI-based process to support the conversation. The design, though, has to properly address eventual conversation breakdowns [10,30].

Achieving effective interaction between users and news chatbots is an intriguing challenge, depending not only on design but also on users’ preferences and interests. More than that, as pointed out by Zamora (2017) [29], the best interaction mechanism should be chosen according to the context; for instance, replacing text-based input, which is more susceptible to errors, with making selections whenever possible (see also Reference [10]). To create a truly inclusive experience, the design should guide the users throughout the navigation, step by step, facilitating their search and news selection process.

Added to all of this, content plays a significant role in the effectiveness of news chatbots. As Roberts and Doyle (2017) [31] point out, crisis-responding organizations are encouraged to build relationships with the local population around questions of data interoperability, data sharing and understanding how their policies might inhibit or affect data sharing and collection. In regard to the use of chatbots for crisis reporting, users need to be able to have quick access to specific information, as well as access to distinct and clear categories of news. Depending on the type of crisis, this information may vary from instructions for access to hospital and medical care facilities, to road safety instructions and/or emergency contact details of relevant authorities.

3. Research Questions (RQs) and Method

Based on the preceding theoretical analysis, the study sought to address the following research questions:

RQ1. Which functions should news chatbots perform when used in crisis reporting?

RQ2. How can news chatbots be designed and implemented to be trustworthy, reliable and acting in the users’ interests when used in crisis reporting?

Based on the ideal features of a chatbot, as earlier analyzed, a news chatbot was created based on the COVID-19 information offered on the web platform of the BBC. The study opted for the implementation of a retrieval based chatbot with predetermined responses based on specific requirements included in this study. Specifically, the scope of the effort was to develop and evaluate a news chatbot that would offer an alternative method of accessing existing information. Moreover, since such a solution is being proposed for crisis situations, a specific one was selected that could be deployed rapidly and does not require sophisticated programming. It is worth noting that chatbot implementations based on natural language interaction can be considered to be overrated for disseminating existing structured (in some sense) information (e.g., symptoms, existing cures, available medication, restriction in movement, etc.).

The specific news organization was selected first because it attracts a global audience through using the English language, and second because it presents not only national information regarding the UK but also global information on the rest of the world, in contrast to other national or local news organizations in Europe that tend to focus more on national/local aspects of the COVID-19 pandemic. This makes it easier for study participants to assess and evaluate the effectiveness of the news chatbot presented, as they are more familiar with the available information.

Once the news chatbot was created, two groups of journalism students (consisting of 45 participants each) were asked to evaluate its performance, both via mobile and computer screens. The questionnaire used for the focus groups’ evaluation is presented in Appendix A. The two focus groups were independent and did not interact with each other. The first group was selected from 2nd-year students attending the BA Program of Journalism in the University of Cyprus; the second group was selected from 2nd-year students attending the BA Program of Journalism in the Aristotle University of Thessaloniki, Greece. All participants in both groups were familiar with news chatbot applications, having earlier attended relevant teaching modules. The ratio of men to women was around 1:1, and their ages ranged from 20 to 24 years old. They were deemed appropriate to evaluate the news chatbot mainly due to their familiarization with relevant applications. After all, this study is not concerned with the overall evaluation of using news chatbots; instead, the main target, as already stated, is to design an effective news chatbot to be used during a crisis situation.

Both focus groups were conducted by using the Microsoft Teams Platform for distance learning courses by the same independent researcher/moderator who guided the interactive conversation. The moderator was accompanied by each group’s instructor, who remained only as an observer throughout the online conversation. The initial questions were based on the specific features of effective news chatbots, as earlier analyzed in the Theoretical Framework. In the next stage of the study, all evaluation comments were categorized regarding functionality, reliability, design and specific features of the news chatbot (see analysis below) and embedded in the final application.

4. The COVINFO Reporter Chatbot: Design and Implementation

The COVINFO Reporter chatbot was developed over a period of two months (March and April 2020) and was carried out in three steps, namely design, implementation and evaluation of the platform. In the first step, which involved platform design, the authors employed existing experience of chatbot development [8], as well as previous related implementations [10,12]. Figure 1 depicts the actions taken during the development of the chatbot. In step 1, the chatbot’s requirements and features were articulated. The second step included the deployment of the COVINFO Reporter, with all the features specified in the first step. The supported conversation was optimized for simplicity of use, aiming at providing all the information requested by users, with minimal conversation steps. During the implementation step, the COVINFO Reporter was evaluated by a small number of experts who completed a series of actions on the chatbot, acting as potential users obtaining information related to COVID-19. Finally, it was extensively used and evaluated by journalism students from Greece and Cyprus. Based on their comments, improvements were introduced, and possible research directions were identified. The evaluation of the chatbot is presented and discussed in the next section.

4.1. Design

As mentioned earlier, the design of the COVINFO Reporter chatbot was based on previous experience [8] in which a chatbot based on the Quriobot plugin was embedded in a news article published on a WordPress platform, as a means of alternative narrative. Although the focus of the current project was on facilitating access to multiple news articles, the main idea was the same in both cases: the deployment of an alternative means of accessing existing information. Of course, one must also consider the urgency of the situation of the COVID-19 pandemic, which forces users to seek important information on how to deal with the crisis, which affects all aspects of their daily lives.

For the process of selecting the appropriate chatbot platform, certain parameters were taken into account. Specifically, the study prompted for a simple chatbot platform that can be embedded in a WordPress Content Management System (CMS), since the majority of the news websites is based on this CMS. Moreover, for future extensions of this effort, the possibility of integrating the chatbot into other social media platforms needed to be supported by the selected chatbot platform. Based on the above, the ManyChat chatbot platform (built primarily of Facebook messenger) was excluded. Furthermore, other established chatbot platforms, like Amazon Lex and Google Dialogflow, do not offer their own plugins for WordPress (some related plugging is offered by third parties). In addition, those platforms are built for natural language interaction. Thus, the Quriobot platform was selected based on the facts that it supports WordPress integration and, also, other social media platforms (Facebook Messenger, Viber, Slack, WhatsApp, Snapchat, etc.) for future extensions and its primary focus is on supporting predetermined chat flows.

The definition of the requirements during the design step was supported by previous chatbot implementations. Piccolo et al. (2018) [10] consider the deployment of a chatbot on Facebook Messenger to help people submit reports of violence and misconduct during the 2017 Kenyan elections. During the design phase of the COVINFO Reporter chatbot, this option was examined but rejected as being limited only to users of one messaging platform. However, this functionality can be deployed at a later stage of the chatbot development, since the chosen implementation platform supports a range of messaging platforms. The BBC has also piloted several conversational chatbots based on pre-scripted material written in a multiple-choice format, mainly focused on special events, e.g., the UK’s general elections [12,32].

Since the scope of this study was to develop a chatbot that would offer an alternative method of accessing existing information, a retrieval-based chatbot with predetermined responses was selected. The BBC website was chosen as the source of the chatbot information, since, among other characteristics given in the RQs and Method section, it has developed a significant number of web articles (text and video) that examine various aspects of the COVID-19 pandemic [33].

4.2. Implementation

The COVINFO Reporter chatbot aims to enhance journalists’ day-to-day workflows. The platform supports the selection by journalists of existing web content from the BBC website, which is made available through the COVINFO Reporter interface and is accessed by users seeking specific information through an alternative narrative. Figure 2 depicts the interactions that take place with involved stakeholders (journalists and users). Journalists create content on the BBC website and also select the articles that can be made available through the COVINFO Reporter chatbot. On the other hand, users employ the COVINFO Reporter interface to access available information.

The Quriobot platform (https://quriobot.com) was selected to deploy the COVINFO Reporter chatbot. The Quriobot is relatively simple to install and use, with development managed via its Control Room (https://control.quriobot.com/). The chatbot is also able to gather information through interaction with users and thus improve interactivity. Although the COVINFO Reporter chatbot was employed through the URL offered by the Quriobot platform, it can also be embedded in a WordPress website, as well as other channels (e.g., Facebook Messenger). The COVINFO Reporter chatbot can be accessed at https://botsrv.com/qb/AUTH/COVINFO-CHATBOT.

The creation of a chatbot is a relatively straightforward process, since the Quriobot platform offers a number of ready-to-use templates that can be adapted to each developer’s needs. It also offers the ability to construct a chatbot from scratch, which consists of a number of steps that define the Quriobot’s behavior. The Quriobot Control Room conversational designer supports the build of proactive conversations, which are supported by conditional rules and different step types. It can also employ smart jumps between questions, based on answers provided by users.

In the case of the COVINFO Reporter chatbot, 11 steps were programmed, as depicted in the flowchart in Figure 3. Initially, there is a welcome message, and the chatbot introduces itself. Then it asks the user if he/she wants to continue. If the answer is “No”, the chatbot terminates. If the reply is “Yes”, the chatbot presents a list of available topic categories, and then, upon selection, the selected category is displayed. Next, the user can select and display a selected news article. Then it asks if the user requires access to other information. If the answer is “Yes”, the chatbot again displays the categories. If the answer is “No”, the conversation is terminated. It is worth noting that all the questions the chatbot is called to answer have predefined answers. Various steps of the conversation with the COVINFO Reporter chatbot are included in Figure 4.

The Quriobot platform supports ongoing modification and update of chatbots by adding more categories or available articles. It can be accessed from both PCs and smartphones. However, one limitation of this feature is that it cannot be programmed to display differently on different device types. Thus, news articles are displayed in a small-size window appropriate to smartphones, which is, as expected, not very convenient for PC users.

5. Evaluation of COVINFO Reporter Chatbot: Findings and Analysis

Drawing from earlier chatbot evaluation studies [6,10,11], the COVINFO Reporter chatbot was assessed in terms of the following characteristics: (a) performance, (b) reliability, (c) functionality, (d) personalization, (e) interactivity, (f) ethics and behavior, and (g) accessibility. Although the two groups of participants did not interact with each other, both came to the same conclusions and offered similar assessments; therefore, their comments are jointly presented based on the earlier mentioned categorization of the chatbot’s characteristics and not on the basis of two separate focus groups.

The performance of the COVINFO Reporter chatbot was examined in regard to its ability to respond in a timely and efficient manner, both via a larger screen (tablet or laptop) and a smaller one (mobile phone). All participants deemed it efficient, and most indicated that, in times of crisis, the chatbot can save time when looking for crucial information: “It helped me save time while looking for much needed information, for example guidelines on medical facilities”, stated M.A. (female, 20 years old, Cyprus). The vast majority of participants stated that there were no differences between the two screen categories used. However, they found the mobile phone screen to be more efficient, perhaps because they were more familiar with it: “All the information is there; either you use it via a computer or via a smartphone, but my opinion is that COVINFO Reporter was constructed for a smartphone” (C.G., female, 22 years old, Cyprus). Several problems were initially detected by participants of both focus groups, focusing mainly on the chatbot’s technical performance. “I detected some ‘technical problems’ in regard to the chatbot’s ability to return to the main screen that need to be fixed; otherwise it would be tiring to navigate through it” (A.L., male, 22 years old, Cyprus); “When using it through a computer, it does not seem to be able to use the full-screen mode, only part of the screen” (I.A., male, 23 years old, Cyprus); “I agree, this could entail further problems with people with visual disabilities and older users” (N.C., female, 21 years old, Cyprus).

Functionality was tested in terms of linguistic accuracy and knowledgeable information offered. Most of the participants agreed that the language used is simple and accurate, and the information offered is useful for everybody during the pandemic crisis: “I found everything I was looking for—the information was filtered in a useful way and the news categories were expressed with simple words” (A.L., male, 23 years old, Greece). Although some participants indicated that they enjoyed the fact that the COVINFO Reporter offers the basic information needed, others argued that more information could be added: “At least another information category has to be added regarding information for the pandemic outside the UK, for people who want to know what is happening in other countries” (G.A., male, 20 years old, Greece; A.E., 21 years, female, Cyprus). An important point was made by several participants regarding the colors used: “the use of red color could eventually make us tired while using the chatbot for [a] longer duration of time” (A.B., female, 22 years old, Greece); “the partial use of white color should be avoided in a news chatbot because it can make users feel bored” (G.A., male, 20 years old, Greece); “definitely the use of white color for the fonts should be avoided in the chatbot responses; it makes it difficult to read the information offered” (G.K., female, 21 years old, Greece).

Reliability was measured in regard both to the content offered and to the chatbot’s proper function. The majority of the participants indicated that the chatbot functioned properly, and the timeframe for providing answers to users was adequately calculated. “It functions properly in all categories tested, and this kept me going for longer than I thought I would have stayed inside the application, definitely better than reading a conventional news website” (S.Ma., male, 21 years old, Greece). Regarding the news content, all participants deemed it reliable and trustworthy, as “it is based on the information offered by the BBC web page; therefore, I consider it reliable enough” (P.P., male, 23 years old, Cyprus); “I trust that the information is reliable, because it comes from a reliable source, since it is the source that guarantees reliability, not the robot application” (C.Ch., female, 22 years old, Cyprus).

Personalization refers to the form and depiction of the chatbot. All participants found the selected form as being “representative”, “reliable” and “affective”. To this end, several students emphasized the fact that the picture selected for the COVINFO Reporter chatbot “encompasses politeness and personality traits that can make it seem more human” (E.A., female, 21 years old, Cyprus), “it looks like a real reporter and uses vivid color, which is very pleasant” (C.C., female, 20 years old, Cyprus), whereas it offers “crucial information in a customized form, and this is important in a crisis situation, for all users” (N.K.Th., female, 22 years old, Greece). All participants argued that the depiction of the chatbot in a humanlike form was not confusing: “it is clear that this is a robot application, although it is depicted in a humanlike form” (N.P., male, 21 years old, Cyprus); “the humanlike picture selected cannot confuse the users; this is clearly a robot we are interacting with, although he looks really friendly and polite, exactly as a reporter should look” (G.I., male, 22 years old, Cyprus).

Interactivity was tested in regard to the chatbot’s ability to easily interact with users. “It was fun and enjoyable to read the news in this format; it really helped me to move on with my next questions”, stated D.Th. (male, 22 years old, Greece). All participants agreed that this was a more enjoyable way to acquire the information they were looking for regarding the pandemic crisis than the way information is offered in a conventional news site: “Even if I am in a hurry and looking for specific information quickly, this way is far more effective, because it is like the chatbot is trying to answer all my questions” (M.A., male, 20 years old, Cyprus).

Ethics and behavior refer to issues regarding users’ privacy and sensitivity toward social concerns. All participants found it positive that no personal data were needed, whereas the information offered was in line with social concerns around an issue as serious as a health pandemic. “I liked the fact that there was no need to state personal data, i.e., my e-mail; it made it easier for me to search the specific information I wanted; for example, if someone I infected looks for medical information, he/she do not need to identify themselves”, stated I.A. (male, 23 years old, Cyprus). “The chatbot provides information for every user of any age, and for me, this indicates social responsibility for all citizens”, argued Ch.K. (female, 21 years old, Greece).

Accessibility was tested in terms of users’ ability to easily access the chatbot and navigate through it. All participants in the study agreed that it was easily accessible and “fun to navigate through it, much more than any conventional news website (M.Z., female, 21 years old, Greece). “It was really easy to access and navigate inside the chatbot; everyone can do it, even older users with limited knowledge regarding chatbots; in fact, the chatbot itself guides you to the information you are looking for” (A.S., female, 21 years old, Greece). “The Q and A process escalates smoothly, and in this way, it is easy to access the specific information you are looking for; it is an easier narrative for telling the news” (P.T., male, 23 years old, Greece). “It is a better narrative when you are looking for emergency information, easier to access and navigate through it” (M.P., female, 22 years, Cyprus).

All expected characteristics, as well as achieved results, are depicted in

Table 1. Expected characteristics and achieved results.

Characteristics	Achieved Results
Performance	Ability to respond in a timely and efficient manner Ability to adjust both to a larger screen (tablet or laptop) and a smaller one (mobile phone)
Functionality	Linguistic accuracy Knowledgeable information Simple language Use of bright colors for the fonts
Reliability	Proper technical function Credible information Identification of information source(s) Proper timeframe for providing answers
Personalization	Humanlike picture Selection of personal characteristics that adhere to a typical reporter
Interactivity	Ability to easily interact with users
Ethics and Behavior	Respect of users’ privacy No personal data shared Information in line with social concerns regarding the pandemic
Accessibility	Easy navigation Easy access Smooth escalation of Q and A process

.

As this analysis has shown, chatbots used for news dissemination in a crisis situation seem to present certain differences in comparison to commercial chatbots. First, according to users’ assessment, they need to be as simple as possible so as any user can access and navigate through the information offered. This is extremely important during a crisis situation, since users need to acquire information fast and easily. In this light, the chatbot’s technical ability to respond in a timely manner is of equal importance during a crisis situation. The second basic difference is related to ethics and behavior. In a crisis situation, social concerns and “sensitive” issues may be related to patients’ identification and personal data publicized. As such, news chatbots used for access to emergency information (e.g., nearby medical facilities, guidance according to medical protocols, etc.) need to be in line with social concerns and ethical boundaries.

This analysis has also shown that the design and development of chatbots used for news dissemination in a crisis situation is rapidly evolving, following two basic factors: first, the latest technological trends, as well as the available technology, both to the developer and to the target audience. For example, while an international news organization can have access to the means and personnel needed to develop a more perplexed chatbot application, a local news entity does not have the means, nor does it employ the specialized personnel, to develop perplexed applications. In this light, news chatbots that are developed to meet urgent needs and audience demands need to be easily designed and managed, following existing development tools. Accordingly, every developer needs to keep in mind the technology available to the target audience. For example, users in countries of the Western world tend to enjoy more advanced technological tools than users in underdeveloped countries, following the existing digital divide.

Second, news chatbots need to be in line with the specific peculiarities of the crisis situation for which they provide information. Not every crisis situation presents similar characteristics to previous crises, even if they are related to the same social sectors. For example, the Great Recession of 2007 was radically different from previous economic crises mankind had to face. Accordingly, the pandemic crisis of 2020 due to COVID-19 was different compared to the SARS (Severe Acute Respiratory Syndrome) pandemic in the early 2000s. As such, users’ needs regarding information and news dissemination may differ, and this has to be taken into consideration during the development of the application.

6. Conclusions

This paper has focused on the design, implementation and evaluation, in terms of effectively fulfilling the social responsibility function of crisis reporting, of a news chatbot used in a crisis situation. In this light, the pandemic crisis of 2020 due to COVID-19 was used as a case study, and the COVINFO Reporter chatbot was developed, which aims to deliver timely and accurate information regarding the crisis. The novelty of the approach is based on the news chatbot’s easy implementation for news organizations, as well as on its ability to effectively deliver crucial information to a wide audience (users) in times of crisis.

Interesting conclusions can be drawn from the findings of the study. There is no doubt that automation is already having a significant impact on journalism and the dissemination of news in general. The introduction of chatbots in the media sector has shown that they can significantly reduce journalists’ workload, allowing them to concentrate on quality, in-depth analysis and reporting [14]. Chatbots can facilitate an alternative narrative that can be customized based on users’ preferences. This is significant in the cases of crisis reporting, where the dissemination of accurate, timely and customized information is very important for the public.

The theoretical study of previous media chatbot projects informed the implementation of the COVINFO Reporter, a working chatbot that disseminates information published by an international media organization. The chatbot was developed on a commercially available chatbot platform (i.e., Quriobot) and can be easily customized and updated. It offers easy and predictive navigation, enabling users to access the information that interests them, without having to navigate through the significant number of webpages that a media organization site usually incudes. Its programming is relatively straightforward and can be easily integrated into the workflow of a typical media organization.

A thorough evaluation of various characteristics (performance, reliability, functionality, personalization, interactivity, ethics and behavior and accessibility) of the COVINFO Reporter chatbot was conducted by two separate groups of participants. The chatbot was positively evaluated in terms of its efficiency, although some participants reported minor technical problems. The preferred platform for accessing it was mobile phones. The majority of the participants was satisfied with the functionality of the chatbot, reporting that the language used was simple and accurate, and the information it provided was useful. The participants agreed that the chatbot was reliable, was functioning properly and provided answers in an acceptable time frame. As far as personalization is concerned, the COVINFO Reporter was reported to be representative, reliable and affective. All participants appreciated the chatbot’s interactivity. No problems were reported in terms of users’ privacy and sensitivity toward social concerns. Finally, all participants agreed that the COVINFO Reporter’s accessibility was very good, and they experienced no problems in navigating the chatbot. Overall, the evaluation of the chatbot was very positive, and the minor problems that were detected were noted and corrected, thus improving its performance.

Future extensions of this work could include additional research into the ways in which chatbots can be employed in crisis reporting, with a focus on their smooth incorporation in the journalistic workflow, with added features and inputs (textual and voice), so as to further assist users looking for specific information. Special attention should be given to chatbots’ ability to collect data from users, thus enabling them to be utilized in crowdsourcing schemes, which can be extremely valuable during crisis situations. However, in this case, the use of a method to prevent the spread of “fake news” (misinformation) would be necessary so as to ensure the reliability of the chatbot and the information disseminated. There exist a variety of techniques to monitor complex systems, in which the average behavior of the whole network is compared to particular nodes. One example is the use of deep learning to detect faults in systems through entropy measurement, as proposed by Martinez-Garcia et al. (2019) [34].

Finally, since this implementation is proposed as an interaction with the general public, interaction in natural language was not the first choice. Nevertheless, the incorporation of a news chatbot that would support natural language interaction is considered to be one of the future extensions of this study.