Bots as Active News Promoters: A Digital Analysis of COVID-19 Tweets

: In this study, we examined the activities of automated social media accounts or bots that tweet or retweet referencing #COVID-19 and #COVID19. From a total sample of over 50 million tweets, we used a mixed method to extract more than 185,000 messages posted by 127 bots. Our ﬁndings show that the majority of these bots tweet, retweet and mention mainstream media outlets, promote health protection and telemedicine, and disseminate breaking news on the number of casualties and deaths caused by COVID-19. We argue that some of these bots are motivated by ﬁnancial incentives, while other bots actively support the survivalist movement by emphasizing the need to prepare for the pandemic and learn survival skills. We only found a few bots that showed some suspicious activity probably due to the fact that our dataset was limited to two hashtags often used by o ﬃ cial health bodies and academic communities.


Introduction
The purpose of this study is to identify bot accounts to understand the nature of messages sent by them on COVID-19. Social media bots have been widely discussed in academic literature as some kind of moral panic mostly in relation to spreading controversial and politically polarized messages or in connection to problematic health bots [1,2]. The findings of this study, however, show that bots that reference COVID-19 mostly mention mainstream media and credible health sources while spreading breaking news on the pandemic or urging people to stay at home. We argue that many bots seem to be spreading news to gain profit through clickbaits or by directing Twitter users to certain websites. We also argue that there are still advantages from using these Twitter bots to inform people about the pandemic risks. Though Twitter bots are different from health chatbots, the results of this study align with previous research on the possible benefits, advantages, or possibilities afforded by the use of official health chatbots [3][4][5][6]. Health chatbots, for example, have shown to be useful in addressing patients' needs in different contexts [7][8][9][10][11][12].
Despite the fact that the intentions and makers of Twitter bots are different from official health chatbots, they are both automated accounts that are active on social media. We argue here that within the context of this COVID-19 pandemic, there are clear similarities but certainly different motivations. WHO, on the one hand, created a health chatbot on Facebook messenger as well as on the messaging app, Viber, to assist in combating misinformation [13,14]. On the other hand, we believe that some of the Twitter bots might be designed to gather Twitter followers and gain popularity, which could be utilized in different ways in generating marketing income. Even the focus on disseminating factual news on COVID-19 can be another way to gain the trust of some Twitter users which is similar to the way mainstream media outlets in the United States originally found objectivity to be financially rewarding because they could use it to attract broader non-partisan audiences [15]. In other words, Nevertheless, social media is not always a positive force. The COVID-19 pandemic has created what is known as an 'infodemic' since a massive amount of misinformation has been spreading on social media at an unprecedented speed [29]. Kouzy et al. [30], for instance, conducted an analysis of 673 tweets and found that medical misinformation and unverifiable content pertaining to the COVID-19 outbreak were widely disseminated as 24.8% of tweets included misinformation and 17.4% contained some unverifiable content. This study provides an early quantification of the magnitude of misinformation spread on Twitter. Similarly, another study conducted by Pulido et al. [31] analyzed the circulation of false and evidence-based information during the COVID-19 pandemic. Their analysis of 1000 tweets showed that false information was tweeted more than science-based evidence or fact-checking information; nevertheless, the circulation of false information was at a lower rate. The importance of urgent interventions to curb the spread of medical misinformation that could jeopardize public health safety was highlighted in this study.
The rapid spread of misinformation on social media fueled panic during the outbreak. Various researches have examined sentiment dynamics on social media discussion pertaining COVID-19. For example, findings of a sentiment analysis of over 20 million posts from Twitter suggested that negative emotions such as fear, anger, and sadness were dominant during the outbreak [32]. Several other studies found that social media has a significant impact on spreading fear and panic during the COVID-19 outbreak that might jeopardize people's psychological well-being, calling for attention and social support on mental health [33,34].
Social media have also perpetuated stereotyping and discrimination against individuals and groups on the basis of their racial identities. After the US presidential reference of the "China Virus" term on 16 March 2020, the rise in tweets referencing "Chinese virus" or "China virus," along with the content of these tweets, indicate that COVID-19 stigma against Asian communities is likely being perpetuated on Twitter [35]. Nevertheless, Asian people are not the only victim of COVID-19 fueled stigmatization. Since older adults have been identified as a group at higher risks of death from COVID-19, a significant proportion of news has covered this topic, which has led to an increase in ageism claiming that COVID-19 is particularly a disease of older people on Twitter [36].
Besides particular social issues of discrimination, various researchers followed an infodemiological approach to map general public concerns on social media during the pandemic. For example, Abd-Alrazaq et al. [37] identified four main public concerns on Twitter in their examination of 167,073 unique tweets: (1) geological origin of the virus, (2) causes leading to the transfer of COVID-19 to humans, (3) impact on people, countries, and the economy, (4) ways of mitigating the risk of infection.
To address the gaps highlighted above on the empirical study of Twitter bots in relation to COVID-19 pandemic, our study attempts to answer the following research questions: RQ1: What is the percentage of social media bots in the examined dataset, RQ2: What is the connection amongst them based on social network analysis, RQ3: What do these bots' tweet about and what are the general sentiments and nature of their messages? RQ4: What are the public discourses on bots?

Methods
We used a mixed approach mostly comprised of several digital methods in this study which align with what Rains [17] mentions in his review of health communication research techniques such as "data acquisition, classification/prediction, text mining, [and] network analysis" (p. 27). First, we collected 50,811,299 tweets and retweets referencing #COVID-19 and #COVID19 for a period of over two months from 12 February until 18 April 2020. We focused on these two hashtags because they are standard terms used by WHO and other official sources. These tweets were sent by 11,706,754 unique users (unique users are those whose Twitter usernames are not repeated in the collected dataset) and this dataset was collected using the Twitter Capture and Analysis Toolset (TCAT) platform that utilizes Twitter public Application programming Interface (API), allowing the collection of a portion of public tweets referencing the above two hashtags [38]. Due to API limitations, the platform sometimes hits the permitted rate limit resulting in brief delays in collecting tweets. In other words, the collection of Information 2020, 11, 461 4 of 13 this social media data was done within the terms and conditions of Twitter guidelines. We then used a Python 3 script to identify the top 1000 most active Twitter users who tweet the most about COVID-19 because bots are known to be very active in spreading messages. For bots' detection, we used the Python version of Botometer because it allows bulk assessment of thousands of accounts unlike the website version that can only be done individually [39]. The tool provides ordinal scores ranging from 0 which is more likely to be human until 5 which is more likely to be bot. After obtaining the tweets sent by the active bots, we found a total of 185,099 tweets and retweets ( Figure 1). We decided to use a score of 3 and above to make sure that the accounts are bots despite the fact that 12 other account names that scored slightly below 3 contained the word bot in them such as @sumanebot, @Bot_Corona_V, and @covid19BotLatam. We then analyzed the bots' tweets using automated sentiment analysis tool called VADER with the use of a Python 3 package [40]. The algorithm calculates the sentiment score of each tweet and measures the mean with values ranging between −1 to +1 (−1 highly negative, +1 highly positive, 0 neutral). All our Python 3 scripts are shared on GitHub. (For more details on our Python 3 scripts, please see the following link: https://github.com/vishukla/covid19-bots) This is followed by using topic modelling to understand the main topics discussed by these bots. Here, we used QDA Miner-WordStat8 commercial software that offers a topic modelling tool built on factor analysis (FA). This software is used because of its practical feature of providing names for the generated topics. The FA approach ranks topics based on the Eigenvalue that is a mathematical linear system, indicating the dominance of certain topics in the text corpus, for the higher this value is, the more dominant the topic is found in the corpora [41]. The formula used is based on calculating the factor loading which identifies the strength of "the relationship of each word to each topic" as " each word wi in the vocabulary V containing all words in a corpus, wwii ∈ VV, ∀ii ∈ {1, . . . , nn}, can be represented as a linear function of m(<n) topics (aka common factors), ttjj ∈ TT, ∀jj ∈ {1, . . . , mm}" [42]. version that can only be done individually [39]. The tool provides ordinal scores ranging from 0 which is more likely to be human until 5 which is more likely to be bot. After obtaining the tweets sent by the active bots, we found a total of 185,099 tweets and retweets ( Figure 1). We decided to use a score of 3 and above to make sure that the accounts are bots despite the fact that 12 other account names that scored slightly below 3 contained the word bot in them such as @sumanebot, @Bot_Corona_V, and @covid19BotLatam. We then analyzed the bots' tweets using automated sentiment analysis tool called VADER with the use of a Python 3 package [40]. The algorithm calculates the sentiment score of each tweet and measures the mean with values ranging between −1 to +1 (−1 highly negative, +1 highly positive, 0 neutral). All our Python 3 scripts are shared on GitHub. (For more details on our Python 3 scripts, please see the following link: https://github.com/vishukla/covid19-bots) This is followed by using topic modelling to understand the main topics discussed by these bots. Here, we used QDA Miner-WordStat8 commercial software that offers a topic modelling tool built on factor analysis (FA). This software is used because of its practical feature of providing names for the generated topics. The FA approach ranks topics based on the Eigenvalue that is a mathematical linear system, indicating the dominance of certain topics in the text corpus, for the higher this value is, the more dominant the topic is found in the corpora [41]. The formula used is based on calculating the factor loading which identifies the strength of "the relationship of each word to each topic" as " each word wi in the vocabulary V containing all words in a corpus, wwii ∈ VV, ∀ii ∈ {1, … , nn}, can be represented as a linear function of m(<n) topics (aka common factors), ttjj ∈ TT, ∀jj ∈ {1, … , mm}" [42]. In addition, we conducted a social network analysis based on usernames and their mentions using Gephi software. To understand the number of online communities that interact among bots, we used the modularity partition method which is built on an algorithm that detects large networks and unfolds a complete hierarchical community structure for the network [43]. Our goal is to understand whether the bots are strongly clustered together and whether they are connected to other similar online communities. We used the following options to generate our graph: OpenOrd spatialization algorithm, modularity class color, and betweenness centrality for the node size. In social network analysis, users are considered "nodes (or actors) and mentions are linked" [44]. Here, we used a Python 3 script to extract mentions from tweets sent by the bot users and arranged the dataset accordingly. The larger nodes show higher connectivity and more interaction in mentioning other users [45]. Our directed network consisted of 29,642 nodes and 42,313 edges, and we also created an interactive high resolution graph (http://gorilladragon.org/COVID_bots/) in order to show all the details of the network. (We would like to thank Dr. Jacob Groshek, an Associate Professor at Kansas State University, for his kind assistance in suggesting a network algorithm and setting up the high resolution graph). Further, we identified some important results like the most recurrent emojis, their frequencies, main categorization In addition, we conducted a social network analysis based on usernames and their mentions using Gephi software. To understand the number of online communities that interact among bots, we used the modularity partition method which is built on an algorithm that detects large networks and unfolds a complete hierarchical community structure for the network [43]. Our goal is to understand whether the bots are strongly clustered together and whether they are connected to other similar online communities. We used the following options to generate our graph: OpenOrd spatialization algorithm, modularity class color, and betweenness centrality for the node size. In social network analysis, users are considered "nodes (or actors) and mentions are linked" [44]. Here, we used a Python 3 script to extract mentions from tweets sent by the bot users and arranged the dataset accordingly. The larger nodes show higher connectivity and more interaction in mentioning other users [45]. Our directed network consisted of 29,642 nodes and 42,313 edges, and we also created an interactive high resolution graph (http://gorilladragon.org/COVID_bots/) in order to show all the details of the network. (We would like to thank Dr. Jacob Groshek, an Associate Professor at Kansas State University, for his kind assistance in suggesting a network algorithm and setting up the high resolution graph). Further, we identified some important results like the most recurrent emojis, their frequencies, main categorization and subcategorization, hashtags, and words used by bots using Python scripts such as EmojiMapper (more information can be found here: https://github.com/JECSand/emoji_unicode_json_mapping). To understand who the bots mostly reference and mention in their tweets, we extracted the most mentioned users using a Python 3 script, and then we conducted a thematic analysis to identify those mentioned users by examining their Twitter profile description and some of their tweets in their timeline. If an account mentions that it is run by a citizen journalist, we code it as such. If there is no description available and an individual's name is used, we code it as personal. We found four main categories among the top 30 most mentioned users which include: (1) health agencies-official bodies, (2) mainstream media, (3) citizen journalists-grassroot organizations, and (4) personal accounts. Further, we compared the top 5000 mentioned users with bots' accounts to see whether these bots reference other similar ones. Finally, we identified English language tweets that reference bots in the overall dataset, and we examined the most retweeted posts to understand the public perspectives on such automated accounts. The main methodological steps we followed can be summarized in the following Chart 1.
Information 2020, 11, x FOR PEER REVIEW 5 of 13 and subcategorization, hashtags, and words used by bots using Python scripts such as EmojiMapper (more information can be found here: https://github.com/JECSand/emoji_unicode_json_mapping). To understand who the bots mostly reference and mention in their tweets, we extracted the most mentioned users using a Python 3 script, and then we conducted a thematic analysis to identify those mentioned users by examining their Twitter profile description and some of their tweets in their timeline. If an account mentions that it is run by a citizen journalist, we code it as such. If there is no description available and an individual's name is used, we code it as personal. We found four main categories among the top 30 most mentioned users which include: (1) health agencies-official bodies, (2) mainstream media, (3) citizen journalists-grassroot organizations, and (4) personal accounts. Further, we compared the top 5000 mentioned users with bots' accounts to see whether these bots reference other similar ones. Finally, we identified English language tweets that reference bots in the overall dataset, and we examined the most retweeted posts to understand the public perspectives on such automated accounts. The main methodological steps we followed can be summarized in the following Chart 1.
Chart 1. The main methodological procedures followed in the study.

Chart 1.
The main methodological procedures followed in the study.

Results
From a total of the 1000 most active users that tweeted the most, we identified 127 active Twitter accounts that scored 3+ out of 5 in their likelihood of being bots; however, there were 40 other accounts that did not have a score either because their accounts were private or were deleted such as @NovelCoronaBot and @covidworldinfo probably because they violated Twitter automation rules. We are not sure, though, whether these deleted accounts are bots or not because Twitter often suspends user accounts for a variety of other reasons like spamming and/or using abusive language [46]. The active bot users sent 185,099 tweets, and the average number of tweets and retweets each bot sent during the time period of the study is 1457. The most active bot user in the whole dataset is @coronavid19_bot (n = 16,484), while other users clearly indicate their bot nature like @StayAtHomeBot and @layoffbot. The majority of other bots do not have the same name features such as @newworldsurvive, @DrCoronavirus, @coronavirusbuzz, and @VirusTimes.
The social network analysis shows 10 major online communities, and the largest is the one in pink (84.15%) which has more than 24,000 nodes followed by the light green one (9.25%) containing about 2000 nodes ( Figure 2). The remaining online communities are smaller than the two main ones.
The social network graph also shows two bot users, @ChalecosAmarill and @AleLRoss198, have large nodes because they are far more interactive in mentioning other users than other ones. Regarding the most referenced users by bots, we found that 5 out of 30 top accounts are bots while two are recently deleted accounts. In terms of their account types, 4 of them are citizen journalists and grassroot organizations and 3 are personal accounts.
As for the top 30 most mentioned users, we found that mainstream media comes first (33.3%) followed by health agencies and official bodies (30%), personal accounts (23.3%), and citizen journalists or grassroot organizations (10%). To further examine these most mentioned users by bots, we ran the Python 3 script for bot detection to see whether they are mostly bots or humans. We limited our focus here to the top 1000 most mentioned users, and we found that 86.9% were more likely to be humans in contrast to 5% that scored between 3-4.9 out of 5, while 7.6% accounts had no scores. Further, the comparison between the 5000 most mentioned users and bots shows that there are only 24 references to other bots, constituting 0.47% of mentioned users.
In relation to topic modelling analysis, four main topics were generated and ranked based on their Eigenvalue including: "total cases" (8.09), "Covid news" (7.22), "prepper bushcraft" (6.76), and "stay at home" (6.42) ( Table 1). Regarding sentiment analysis, the mean results indicate (0.016212) score with a standard deviation of (0.343864) and variance of (0.118243). As for the most recurrent words, we found that the top seven ones are used 116,112 in total in the tweets, including "new", "cases", "total", "confirmed", "worldwide", "totaling", and "death". The bots accounts sent 3136 emojis often including a hashtag #covid19memes (n = 3038). The most recurrent emojis include As for the most recurrent words, we found that the top seven ones are used 116,112 in total in the tweets, including "new", "cases", "total", "confirmed", "worldwide", "totaling", and "death". The bots accounts sent 3136 emojis often including a hashtag #covid19memes (n = 3038). The most recurrent emojis include (confirmed cases, n = 8529), (deaths, n = 4773), (recovered, n = 3685), ☣ (virus, n = 7932) as well as other related emojis like (virus, n = 4181), (Face mask, n = 2076), ☠ (death, n = 3278), and (SOS, n = 1135). The sequence of emojis gives another insight on the conveyed meaning of these information items; for example, we find that country emoji flags constitute some of the top 20 most recurrent sequences, indicating the ongoing news updates on the number of COVID's patients, deaths, and recoveries (See Table 2). We also find other emoji sequences like ☣ (disseminating news on the virus, = 678), ☣ (disseminating videos on the virus, = 666), (n = 110), (please wash your hands with soap and water, n = 39), and (sick people around the world, n = 12). In terms of emoji categorization, the results show that emoji symbols are the most frequent ones followed by smileys and people, travel, places and flags, objects, food and drink, and animals and nature. In addition, the list of the top 20 emojis subcategories contain important cue like warnings, sick face, and negative face emojis (Table 3). Finally, we identified 3232 tweets posted by Twitter users that referenced bots in English language.
(confirmed cases, n = 8529), As for the most recurrent words, we found that the top seven ones are used 116,112 in total in the tweets, including "new", "cases", "total", "confirmed", "worldwide", "totaling", and "death". The bots accounts sent 3136 emojis often including a hashtag #covid19memes (n = 3038). The most recurrent emojis include (confirmed cases, n = 8529), (deaths, n = 4773), (recovered, n = 3685), ☣ (virus, n = 7932) as well as other related emojis like (virus, n = 4181), (Face mask, n = 2076), ☠ (death, n = 3278), and (SOS, n = 1135). The sequence of emojis gives another insight on the conveyed meaning of these information items; for example, we find that country emoji flags constitute some of the top 20 most recurrent sequences, indicating the ongoing news updates on the number of COVID's patients, deaths, and recoveries (See Table 2). We also find other emoji sequences like ☣ (disseminating news on the virus, = 678), ☣ (disseminating videos on the virus, = 666), (n = 110), (please wash your hands with soap and water, n = 39), and (sick people around the world, n = 12). In terms of emoji categorization, the results show that emoji symbols are the most frequent ones followed by smileys and people, travel, places and flags, objects, food and drink, and animals and nature. In addition, the list of the top 20 emojis subcategories contain important cue like warnings, sick face, and negative face emojis (Table 3). Finally, we identified 3232 tweets posted by Twitter users that referenced bots in English language.
(deaths, n = 4773), As for the most recurrent words, we found that the top seven ones are used 116,112 in total in the tweets, including "new", "cases", "total", "confirmed", "worldwide", "totaling", and "death". The bots accounts sent 3136 emojis often including a hashtag #covid19memes (n = 3038). The most recurrent emojis include (confirmed cases, n = 8529), (deaths, n = 4773), (recovered, n = 3685), ☣ (virus, n = 7932) as well as other related emojis like (virus, n = 4181), (Face mask, n = 2076), ☠ (death, n = 3278), and (SOS, n = 1135). The sequence of emojis gives another insight on the conveyed meaning of these information items; for example, we find that country emoji flags constitute some of the top 20 most recurrent sequences, indicating the ongoing news updates on the number of COVID's patients, deaths, and recoveries (See Table 2). We also find other emoji sequences like ☣ (disseminating news on the virus, = 678), ☣ (disseminating videos on the virus, = 666), (n = 110), (please wash your hands with soap and water, n = 39), and (sick people around the world, n = 12). In terms of emoji categorization, the results show that emoji symbols are the most frequent ones followed by smileys and people, travel, places and flags, objects, food and drink, and animals and nature. In addition, the list of the top 20 emojis subcategories contain important cue like warnings, sick face, and negative face emojis (Table 3). Finally, we identified 3232 tweets posted by Twitter users that referenced bots in English language.
(recovered, n = 3685), As for the most recurrent words, we found that the top seven ones are used 116,112 in total in the tweets, including "new", "cases", "total", "confirmed", "worldwide", "totaling", and "death". The bots accounts sent 3136 emojis often including a hashtag #covid19memes (n = 3038). The most recurrent emojis include (confirmed cases, n = 8529), (deaths, n = 4773), (recovered, n = 3685), ☣ (virus, n = 7932) as well as other related emojis like (virus, n = 4181), (Face mask, n = 2076), ☠ (death, n = 3278), and (SOS, n = 1135). The sequence of emojis gives another insight on the conveyed meaning of these information items; for example, we find that country emoji flags constitute some of the top 20 most recurrent sequences, indicating the ongoing news updates on the number of COVID's patients, deaths, and recoveries (See Table 2). We also find other emoji sequences like ☣ (disseminating news on the virus, = 678), ☣ (disseminating videos on the virus, = 666), (n = 110), (please wash your hands with soap and water, n = 39), and (sick people around the world, n = 12). In terms of emoji categorization, the results show that emoji symbols are the most frequent ones followed by smileys and people, travel, places and flags, objects, food and drink, and animals and nature. In addition, the list of the top 20 emojis subcategories contain important cue like warnings, sick face, and negative face emojis (Table 3). Finally, we identified 3232 tweets posted by Twitter users that referenced bots in English language.
(virus, n = 7932) as well as other related emojis like 3. As for the most recurrent words, we found that the top seven ones are used 116,112 in total in the tweets, including "new", "cases", "total", "confirmed", "worldwide", "totaling", and "death". The bots accounts sent 3136 emojis often including a hashtag #covid19memes (n = 3038). The most recurrent emojis include (confirmed cases, n = 8529), (deaths, n = 4773), (recovered, n = 3685), ☣ (virus, n = 7932) as well as other related emojis like (virus, n = 4181), (Face mask, n = 2076), ☠ (death, n = 3278), and (SOS, n = 1135). The sequence of emojis gives another insight on the conveyed meaning of these information items; for example, we find that country emoji flags constitute some of the top 20 most recurrent sequences, indicating the ongoing news updates on the number of COVID's patients, deaths, and recoveries (See Table 2). We also find other emoji sequences like ☣ (disseminating news on the virus, = 678), ☣ (disseminating videos on the virus, = 666), (n = 110), (please wash your hands with soap and water, n = 39), and (sick people around the world, n = 12). In terms of emoji categorization, the results show that emoji symbols are the most frequent ones followed by smileys and people, travel, places and flags, objects, food and drink, and animals and nature. In addition, the list of the top 20 emojis subcategories contain important cue like warnings, sick face, and negative face emojis (Table 3). Finally, we identified 3232 tweets posted by Twitter users that referenced bots in English language.  As for the most recurrent words, we found that the top seven ones are used 116,112 in total in the tweets, including "new", "cases", "total", "confirmed", "worldwide", "totaling", and "death". The bots accounts sent 3136 emojis often including a hashtag #covid19memes (n = 3038). The most recurrent emojis include (confirmed cases, n = 8529), (deaths, n = 4773), (recovered, n = 3685), ☣ (virus, n = 7932) as well as other related emojis like (virus, n = 4181), (Face mask, n = 2076), ☠ (death, n = 3278), and (SOS, n = 1135). The sequence of emojis gives another insight on the conveyed meaning of these information items; for example, we find that country emoji flags constitute some of the top 20 most recurrent sequences, indicating the ongoing news updates on the number of COVID's patients, deaths, and recoveries (See Table 2). We also find other emoji sequences like ☣ (disseminating news on the virus, = 678), ☣ (disseminating videos on the virus, = 666), (n = 110), (please wash your hands with soap and water, n = 39), and (sick people around the world, n = 12). In terms of emoji categorization, the results show that emoji symbols are the most frequent ones followed by smileys and people, travel, places and flags, objects, food and drink, and animals and nature. In addition, the list of the top 20 emojis subcategories contain important cue like warnings, sick face, and negative face emojis (Table 3). Finally, we identified 3232 tweets posted by Twitter users that referenced bots in English language.
(Face mask, n = 2076), As for the most recurrent words, we found that the top seven ones are used 116,112 in total in the tweets, including "new", "cases", "total", "confirmed", "worldwide", "totaling", and "death". The bots accounts sent 3136 emojis often including a hashtag #covid19memes (n = 3038). The most recurrent emojis include (confirmed cases, n = 8529), (deaths, n = 4773), (recovered, n = 3685), ☣ (virus, n = 7932) as well as other related emojis like (virus, n = 4181), (Face mask, n = 2076), ☠ (death, n = 3278), and (SOS, n = 1135). The sequence of emojis gives another insight on the conveyed meaning of these information items; for example, we find that country emoji flags constitute some of the top 20 most recurrent sequences, indicating the ongoing news updates on the number of COVID's patients, deaths, and recoveries (See Table 2). We also find other emoji sequences like ☣ (disseminating news on the virus, = 678), ☣ (disseminating videos on the virus, = 666), (n = 110), (please wash your hands with soap and water, n = 39), and (sick people around the world, n = 12). In terms of emoji categorization, the results show that emoji symbols are the most frequent ones followed by smileys and people, travel, places and flags, objects, food and drink, and animals and nature. In addition, the list of the top 20 emojis subcategories contain important cue like warnings, sick face, and negative face emojis (Table 3). Finally, we identified 3232 tweets posted by Twitter users that referenced bots in English language.

3.
Prepper As for the most recurrent words, we found that the top seven ones are used 116,112 in total in the tweets, including "new", "cases", "total", "confirmed", "worldwide", "totaling", and "death". The bots accounts sent 3136 emojis often including a hashtag #covid19memes (n = 3038). The most recurrent emojis include (confirmed cases, n = 8529), (deaths, n = 4773), (recovered, n = 3685), ☣ (virus, n = 7932) as well as other related emojis like (virus, n = 4181), (Face mask, n = 2076), ☠ (death, n = 3278), and (SOS, n = 1135). The sequence of emojis gives another insight on the conveyed meaning of these information items; for example, we find that country emoji flags constitute some of the top 20 most recurrent sequences, indicating the ongoing news updates on the number of COVID's patients, deaths, and recoveries (See Table 2). We also find other emoji sequences like ☣ (disseminating news on the virus, = 678), ☣ (disseminating videos on the virus, = 666), (n = 110), (please wash your hands with soap and water, n = 39), and (sick people around the world, n = 12). In terms of emoji categorization, the results show that emoji symbols are the most frequent ones followed by smileys and people, travel, places and flags, objects, food and drink, and animals and nature. In addition, the list of the top 20 emojis subcategories contain important cue like warnings, sick face, and negative face emojis (Table 3). Finally, we identified 3232 tweets posted by Twitter users that referenced bots in English language.
(SOS, n = 1135). The sequence of emojis gives another insight on the conveyed meaning of these information items; for example, we find that country emoji flags constitute some of the top 20 most recurrent sequences, indicating the ongoing news updates on the number of COVID's patients, deaths, and recoveries (See Table 2). We also find other emoji sequences like  As for the most recurrent words, we found that the top seven ones are used 116,112 in total in the tweets, including "new", "cases", "total", "confirmed", "worldwide", "totaling", and "death". The bots accounts sent 3136 emojis often including a hashtag #covid19memes (n = 3038). The most recurrent emojis include (confirmed cases, n = 8529), (deaths, n = 4773), (recovered, n = 3685), ☣ (virus, n = 7932) as well as other related emojis like (virus, n = 4181), (Face mask, n = 2076), ☠ (death, n = 3278), and (SOS, n = 1135). The sequence of emojis gives another insight on the conveyed meaning of these information items; for example, we find that country emoji flags constitute some of the top 20 most recurrent sequences, indicating the ongoing news updates on the number of COVID's patients, deaths, and recoveries (See Table 2). We also find other emoji sequences like ☣ (disseminating news on the virus, = 678), ☣ (disseminating videos on the virus, = 666), (n = 110), (please wash your hands with soap and water, n = 39), and (sick people around the world, n = 12). In terms of emoji categorization, the results show that emoji symbols are the most frequent ones followed by smileys and people, travel, places and flags, objects, food and drink, and animals and nature. In addition, the list of the top 20 emojis subcategories contain important cue like warnings, sick face, and negative face emojis (Table 3). Finally, we identified 3232 tweets posted by Twitter users that referenced bots in English language.  As for the most recurrent words, we found that the top seven ones are used 116,112 in total in the tweets, including "new", "cases", "total", "confirmed", "worldwide", "totaling", and "death". The bots accounts sent 3136 emojis often including a hashtag #covid19memes (n = 3038). The most recurrent emojis include (confirmed cases, n = 8529), (deaths, n = 4773), (recovered, n = 3685), ☣ (virus, n = 7932) as well as other related emojis like (virus, n = 4181), (Face mask, n = 2076), ☠ (death, n = 3278), and (SOS, n = 1135). The sequence of emojis gives another insight on the conveyed meaning of these information items; for example, we find that country emoji flags constitute some of the top 20 most recurrent sequences, indicating the ongoing news updates on the number of COVID's patients, deaths, and recoveries (See Table 2). We also find other emoji sequences like ☣ (disseminating news on the virus, = 678), ☣ (disseminating videos on the virus, = 666), (n = 110), (please wash your hands with soap and water, n = 39), and (sick people around the world, n = 12). In terms of emoji categorization, the results show that emoji symbols are the most frequent ones followed by smileys and people, travel, places and flags, objects, food and drink, and animals and nature. In addition, the list of the top 20 emojis subcategories contain important cue like warnings, sick face, and negative face emojis (Table 3). Finally, we identified 3232 tweets posted by Twitter users that referenced bots in English language.
(please wash your hands with soap and water, n = 39), and As for the most recurrent words, we found that the top seven ones are used 116,112 in total in the tweets, including "new", "cases", "total", "confirmed", "worldwide", "totaling", and "death". The bots accounts sent 3136 emojis often including a hashtag #covid19memes (n = 3038). The most recurrent emojis include (confirmed cases, n = 8529), (deaths, n = 4773), (recovered, n = 3685), ☣ (virus, n = 7932) as well as other related emojis like (virus, n = 4181), (Face mask, n = 2076), ☠ (death, n = 3278), and (SOS, n = 1135). The sequence of emojis gives another insight on the conveyed meaning of these information items; for example, we find that country emoji flags constitute some of the top 20 most recurrent sequences, indicating the ongoing news updates on the number of COVID's patients, deaths, and recoveries (See Table 2). We also find other emoji sequences like ☣ (disseminating news on the virus, = 678), ☣ (disseminating videos on the virus, = 666), (n = 110), (please wash your hands with soap and water, n = 39), and (sick people around the world, n = 12). In terms of emoji categorization, the results show that emoji symbols are the most frequent ones followed by smileys and people, travel, places and flags, objects, food and drink, and animals and nature. In addition, the list of the top 20 emojis subcategories contain important cue like warnings, sick face, and negative face emojis (Table 3). Finally, we identified 3232 tweets posted by Twitter users that referenced bots in English language.

Prepper bushcraft
(sick people around the world, n = 12). In terms of emoji categorization, the results show that emoji symbols are the most frequent ones followed by smileys and people, travel, places and flags, objects, food and drink, and animals and nature. In addition, the list of the top 20 emojis subcategories contain important cue like warnings, sick face, and negative face emojis (Table 3). Finally, we identified 3232 tweets posted by Twitter users that referenced bots in English language.   As for the top 30 most mentioned users, we found that mainstream m followed by health agencies and official bodies (30%), personal accoun journalists or grassroot organizations (10%). To further examine these most m we ran the Python 3 script for bot detection to see whether they are most limited our focus here to the top 1000 most mentioned users, and we found likely to be humans in contrast to 5% that scored between 3-4.9 out of 5, whi scores. Further, the comparison between the 5000 most mentioned users an are only 24 references to other bots, constituting 0.47% of mentioned users.
In relation to topic modelling analysis, four main topics were generate their Eigenvalue including: "total cases" (8.09)," Covid news" (7.22), "prepp "stay at home" (6.42) ( Table 1). Regarding sentiment analysis, the mean re score with a standard deviation of (0.343864) and variance of (0.118243). As for the most recurrent words, we found that the top seven ones are the tweets, including "new", "cases", "total", "confirmed", "worldwide", " The bots accounts sent 3136 emojis often including a hashtag #covid19mem recurrent emojis include (confirmed cases, n = 8529), (deaths, n = 4773), ☣ (virus, n = 7932) as well as other related emojis like (virus, n = 4181), ☠ (death, n = 3278), and (SOS, n = 1135). The sequence of emojis gives conveyed meaning of these information items; for example, we find th constitute some of the top 20 most recurrent sequences, indicating the ongoi number of COVID's patients, deaths, and recoveries (See Table 2). We also fin like ☣ (disseminating news on the virus, = 678), ☣ (disseminating vid (n = 110), (please wash your hands with soap and water, n = people around the world, n = 12). In terms of emoji categorization, the results s are the most frequent ones followed by smileys and people, travel, places and drink, and animals and nature. In addition, the list of the top 20 emoji important cue like warnings, sick face, and negative face emojis (Table 3). Fin tweets posted by Twitter users that referenced bots in English language. As for the top 30 most mentioned users, we found that mainstream media comes first (33.3%) followed by health agencies and official bodies (30%), personal accounts (23.3%), and citizen journalists or grassroot organizations (10%). To further examine these most mentioned users by bots, we ran the Python 3 script for bot detection to see whether they are mostly bots or humans. We limited our focus here to the top 1000 most mentioned users, and we found that 86.9% were more likely to be humans in contrast to 5% that scored between 3-4.9 out of 5, while 7.6% accounts had no scores. Further, the comparison between the 5000 most mentioned users and bots shows that there are only 24 references to other bots, constituting 0.47% of mentioned users.
In relation to topic modelling analysis, four main topics were generated and ranked based on their Eigenvalue including: "total cases" (8.09)," Covid news" (7.22), "prepper bushcraft" (6.76), and "stay at home" (6.42) ( Table 1). Regarding sentiment analysis, the mean results indicate (0.016212) score with a standard deviation of (0.343864) and variance of (0.118243). As for the most recurrent words, we found that the top seven ones are used 116,112 in total in the tweets, including "new", "cases", "total", "confirmed", "worldwide", "totaling", and "death". The bots accounts sent 3136 emojis often including a hashtag #covid19memes (n = 3038). The most recurrent emojis include (confirmed cases, n = 8529), (deaths, n = 4773), (recovered, n = 3685), ☣ (virus, n = 7932) as well as other related emojis like (virus, n = 4181), (Face mask, n = 2076), ☠ (death, n = 3278), and (SOS, n = 1135). The sequence of emojis gives another insight on the conveyed meaning of these information items; for example, we find that country emoji flags constitute some of the top 20 most recurrent sequences, indicating the ongoing news updates on the number of COVID's patients, deaths, and recoveries (See Table 2). We also find other emoji sequences like ☣ (disseminating news on the virus, = 678), ☣ (disseminating videos on the virus, = 666), (n = 110), (please wash your hands with soap and water, n = 39), and (sick people around the world, n = 12). In terms of emoji categorization, the results show that emoji symbols are the most frequent ones followed by smileys and people, travel, places and flags, objects, food and drink, and animals and nature. In addition, the list of the top 20 emojis subcategories contain important cue like warnings, sick face, and negative face emojis (Table 3). Finally, we identified 3232 tweets posted by Twitter users that referenced bots in English language.   (Figure 2). The remaining online communities are smaller than the two main ones.
The social network graph also shows two bot users, @ChalecosAmarill and @AleLRoss198, have large nodes because they are far more interactive in mentioning other users than other ones. Regarding the most referenced users by bots, we found that 5 out of 30 top accounts are bots while two are recently deleted accounts. In terms of their account types, 4 of them are citizen journalists and grassroot organizations and 3 are personal accounts.

Discussion
To answer the study's research questions, we found that 12.7% of the top 1000 most active Twitter users who reference #COVID-19 and #COVID19 are bots, yet this percentage increases if we incorporate some of the deleted accounts as well as those that slightly scored below three out of five. We believe the percentage of bots identified in this study is an important indicator of the presence and possible influence of these automated accounts on social media. What is interesting, though, is that two of the active accounts initially identified in our dataset which both sent 2049 tweets did not yield a bot score because they did not have any tweets available in their timelines. Previous research shows that some problematic social media users intentionally delete their posts to avoid detection or removal by social media platforms [47]. The qualitative examination of one of these accounts, which sent 685 messages, shows praise for the COVID-19 actions taken by China including spreading unverified information about the alleged effectiveness of traditional Chinese medicine in curing the virus. Almost all the retweets were meant to amplify official Chinese sources like @ChinaEUMission, @Chinacultureorg, and @spokespersonCHN as well as Chinese mainstream media like @ChinaDaily, @CGTNofficial, and @XHNews.
To understand whom these bots mostly reference and mention, we found that they mostly cite mainstream media channels from different countries such as ABSCBNNews (Philippines), El Comercio (Ecuador), and Al Etihad (UAE) followed by health agencies and official bodies like the WHO, CDC, and Dr. Tedros Ghebreyesus. These categories are followed by personal accounts and citizen journalists or grassroot organizations. The prominence of citing, retweeting, and mentioning mainstream media outlets and official health agencies show that the bots are mostly amplifying news on COVID-19. Though these bots are obviously profiting from directing users to their websites, we think that they can still be useful in complementing the information activities of credible sources because they can further disseminate news on COVID-19.
The sentiment analysis of the bots' tweets show that the mean is neutral since the tweets mostly focus on disseminating news and updated figures that do not contain sensationalism or extreme negative or positive sentiments. The emoji subcategorization provides further insight into the sentiment expressed by bots, for Table 3 shows that there is some balance between neutral, positive, and negative emojis like warnings and sick as well as negative faces.
Further, the topic modelling results show that bots that reference COVID-19 are mostly disseminating updated news on the pandemic which is evident from the first and second topics. This is corroborated with the evidence gathered from the most frequent words like "cases", "total", "confirmed", and "worldwide" as well as some of the most used hashtags such as #covid19news (n = 3103) and emojis. We argue here that the purpose of using emojis and their sequences is to further attract the attention of Twitter users due to their appealing pictorial non-verbal communication qualities. They function as a complementary message to the news on COVID-19 such as conveying ). The fourth salient topic is a call to stay at home which is borrowed from the advice given by numerous health agencies and official bodies around the world. This is evident in the use the most frequent hashtags such as #Stayhome, #WashYourHands, #StaySafe, #stayathome, #protectyourselfandyourfamily, and #StayHomeSaveLives (totally used 12,759). However, the third topic which focuses on survival skills like "prepper bushcraft" is quite different, for it supports the survivalism subculture movement that is built on the belief that a natural disaster, war, or pandemic is inevitable, so people need to prepare by purchasing enough goods and learning bushcraft skills to ensure their survival [48]. The presence of this frequent topic is also corroborated with the evidence collected from examining the most frequent hashtags like #survival, #bushcraft, and #prepper used 21,215 times in total.
Though it is not among the most frequent topics, the practice of telemedicine (n = 725) during pandemic times is highlighted in relation to promoting the services of an Indian medical clinic.
In addition, the social network analysis results show that these bots neither function in a coordinated way nor focus on one community, for there are 10 different online communities that are not strongly connected or clustered together, denoting the various mentions and scattered audiences that they target. These findings are corroborated with the evidence cited above on the high percentage of human accounts mentioned by the bots examined in this study (86.9%) as well as the very low percentage of bots referencing similar ones (0.47%).
Finally and to answer the fourth research question, the public discourses on automated bots are mostly focused on the affordances of health chatbots around the world such as launching a botchat called Yani in the Philippines, a WhatsApp chatbot to disseminate "reliable information and rapid testing diagnostics" in Senegal, a telegram bot providing "verified claims" on COVID-19 in India or the "World's 1st Multilingual AI-Bot" on COVID-19 in Pakistan. The Kansas Department of Health and Environment and the Missouri Department of Health and Senior Services in the USA, for example, tweeted about launching chatbots to answer COVID-19-related questions. On the other hand, the Twitter public expressed a few concerns regarding harassment, the politicization of COVID-19, and the possible spread of disinformation by bots allegedly run by the Chinese and Russian governments.
Conceptually, we believe the findings of this study can be useful in developing theory because the majority of previous studies on social media bots focus on their nefarious disinformation functions especially in connection to research on health communication [1,2]. We think that this conceptualization is limited because it does not take into account the possible financial incentives behind using Twitter bots that disseminate factual news rather than disinformation, and it does not provide a complete picture of the bots' useful implementation in public health.

Conclusions
Despite that some literature on social media bots highlight the controversial and anti-social nature of automated accounts, the findings of this study show that the majority of bots spread news on and awareness of COVID-19 risks while citing and referencing mainstream media outlets and credible health sources. We argue that there might be financial incentives behind designing some of these bots. However, and if monitored and updated with credible information by health agencies themselves, we believe that bots can be useful during health crises due to their efficiency and speed in spreading valuable information, some of which is crucial for public health.
It is imperative to highlight some of the limitations of this study. First, the tweets and retweets that we collected only referenced two hashtags #COVID-19 and #COVID19 which are the scientific names of the pandemic because we were interested to see how this community discusses the pandemic. These terms are normally used by an online community that is expected to be more knowledgeable about the virus and they use or rely on official sources. This type of community can be different from other online communities that prefer using other popular terms like #coronavirus or some controversially popular hashtags like #Wuhanvirus or #Chinesevirus. In other words, the findings of our study are only limited to the online community that prefers using the mainstream scientific terms, which might explain why we have not seen many bots that disseminated disinformation or other problematic content amongst the most active bot users. Second, it is important to know the sources of these bots and their possible financial incentives, yet such information cannot be easily obtained, and this remains a major limitation. In this study, for example, we found evidence regarding one suspicious Twitter account that provides support for the Chinese government's narratives regarding COVID-19. Another limitation of the study is the platform choice as we focused on Twitter alone, yet there are many other social media platforms that allow using bots such as Telegram.