A Data-Driven Framework for Coding the Intent and Extent of Political Tweeting, Disinformation, and Extremism

Abstract

Disinformation campaigns on online social networks (OSNs) in recent years have underscored democracy’s vulnerability to such operations and the importance of identifying such operations and dissecting their methods, intents, and source. This paper is another milestone in a line of research on political disinformation, propaganda, and extremism on OSNs. A total of 40,000 original Tweets (not re-Tweets or Replies) related to the U.S. 2020 presidential election are collected. The intent, focus, and political affiliation of these political Tweets are determined through multiple discussions and revisions. There are three political affiliations: rightist, leftist, and neutral. A total of 171 different classes of intent or focus are defined for Tweets. A total of 25% of Tweets were left out while defining these classes of intent. The purpose is to assure that the defined classes would be able to cover the intent and focus of unseen Tweets (Tweets that were not used to determine and define these classes) and no new classes would be required. This paper provides these classes, their definition and size, and example Tweets from them. If any information is included in a Tweet, its factuality is verified through valid news sources and articles. If any opinion is included in a Tweet, it is determined that whether or not it is extreme, through multiple discussions and revisions. This paper provides analytics with regard to the political affiliation and intent of Tweets. The results show that disinformation and extreme opinions are more common among rightists Tweets than leftist Tweets. Additionally, Coronavirus pandemic is the topic of almost half of the Tweets, where 25.43% of Tweets express their unhappiness with how Republicans have handled this pandemic.

1. Introduction

Online social networks (OSNs) were established in the early 2000s, with the objective to make profit out of advertisement. With millions of users sharing their personal information and opinions on various topics, the information value of OSNs was soon discovered. With the possibility to post any content free of fact-checking filters [1] or source verification [2] with no legal consequences, comment on other posts, avoid face-to-face interaction, and engage anonymously, OSNs have become the ground for spreading disinformation and extreme opinions by individuals, organizations, and states with various goals.

Misinformation refers to false information, which misleads readers, and its unintentional spread. Disinformation refers to false information being deliberately spread to serve an entity’s goals by dishonestly misleading readers. For instance, satirical false information which has no intent to mislead or deceive readers in its context and is unlikely to be misperceived as factual does not fit our definition of false information. People’s vulnerability to believe false information on OSNs and other questionable sources stems from two factors: confirmation bias, which means people prefer information that is in line with their existing views [3], and naïve realism, which means people tend to deem their perception of reality as the only accurate view and label those who disagree as biased, irrational, or uninformed [4].

From east to west, disinformation operations on OSNs are being applied by both state and rogue actors—sometimes applied by individuals or parties to affect internal matters inside a country and sometimes to interfere in the internal affairs of another country; sometimes applied to influence people’s minds on a policy, individual, or group, and sometimes to divide people, and sometimes to cause violence [5] and wars. Regardless of its sponsors, extent, and objectives, disinformation operations on OSNs have proven effective, and proportional to its alternatives, low cost. Their success is rooted in replacing the single truth, which is bitter to many, with multiple versions of falsehood, tailored to different tastes. They capitalize on people’s desire to believe what they like rather than the truth. Disinformation operations target democracy’s soft spot at its heart— free speech.

Disinformation operations were conducted by domestic actors to: impugn President Obama’s religion and birthplace [6,7], negate public opinion on the Affordable Care Act by spreading baseless claims, such as death panels [8,9], misrepresent the evidence with regard to Iraq’s role in the 9/11 attack and its mass destruction weapons [10,11], and undermine the factuality of climate change [12,13]. The impact of political disinformation operations on OSNs has been harshly felt in the U.S. in recent few years. Russia’s disinformation operation on OSNs, in 2016, to cast doubt on the credibility of the U.S. political system and federal election and to polarize U.S. citizens [14,15], was one of the most impactful of its type. Three years later, in 2019, they launched another disinformation operation to blame their previous operation on Ukraine. This operation’s shattering influence reached not only main stream media but the Congress and the White House. In 2018, they spread misinformation on OSNs that Obama’s administration abused its power to surveil Trump’s campaign and that the FBI and deep state are attempting to delegitimize Trump’s presidency [16]. The embrace of this misinformation by a large number of citizens and politicians is still negatively impacting U.S. democratic principles. In 2018, Cambridge Analytica sent personalized political messages to U.S. citizens to influence their opinion about the United States’ internal policies [16]. Misinformation operations have been utilized by other countries as well [5,17,18].

Desouza et al. [16] showed that the volume and diversity of the information on OSNs, big data analysis, extracting personal information, and studying people’s behaviors are what are taken advantage of to persuade OSN users toward certain political views. It has also been shown that recommendation algorithms on OSNs lead people to political extremism by feeding them more of the same content [1,19,20]. Moreover, recommendation algorithms on OSNs work to the benefit of hyperactive users [21,22,23], who excessively amplify or discredit certain content on OSNs [24]. By overcontributing to OSNs, these users represent a biased picture of public opinion on various political matters.

All this highlights democracy’s vulnerability to disinformation operations and the importance of studies on identifying such operations and dissecting their methods, intents, and source. This study develops a data-driven framework for classifying the intent of political Tweets, along with analytics regarding their factuality, extremism, and political affiliation. This not only provides insight into OSNs’ role in politics and election and enlightens voters, politicians, and OSNs’ managers in this regard, but also establishes the foundation for different mechanisms that OSNs might take to provide more specific analytics regarding political discourse and flag suspicious activities. Applying machine learning to detect the intent and party affiliation of political Tweets and whether they contain disinformation [25,26,27] or extremism is based on the assumption that these classes are already defined. Our framework establishes a detailed foundation for these automatic approaches.

2. Data Collection and Manual Classification

A total of 1,349,373 original Tweets (not re-Tweets or Replies), containing at least one of the following four terms: Trump, Biden, Democrats, and Republicans, have been collected by our server in real time from the beginning of April 2020 to the end of January 2021. Collecting Tweets in real time is essential for our study, since Tweets containing disinformation or extreme opinions are sometimes removed from the Twitter platform. Out of all original Tweets, 1,048,233 (78%) contain the term Trump, 325,215 (24%) contain the term Biden, 61,810 (5%) contain the term “Republicans”, and 61,811 (5%) contain the term “Democrats”. These Tweets were published by 605,225 different Twitter accounts, where 445,815 of these Twitter accounts posted only one Tweet and 73,512 of them posted only two Tweets. In other words, 756,534 Tweets (56% of all original Tweets) were generated by only 85,898 Twitter accounts. The account publishing the largest number of original Tweets is @TomthunkitsMind with 2787 (0.2%) Tweets, followed by @TheHill with 1385 Tweets, @rogue_corq with 818 Tweets, and @randomtrump1 with 788 Tweets.

The most retweeted is a Tweet posted from Joe Biden’s official Twitter account on September 19, 2020 containing a video clip of Donald Trump saying that “If I lose to him [Joe Biden], I don’t know what I’m gonna do, I will never speak to you again, you’ll never see me again”. The Tweet with the highest number of replies is posted by Ronna McDaniel, the GOP Chairwoman, on September 13, 2020: “Joe Biden can’t run from his disastrous record responding to the Coronavirus. The truth hurts, Joe!” Additionally,

Table 1. The most frequent terms and phrases among Tweets.

Term	Trump	Biden	President	Democrat	Republican	Election	Vote	Coronavirus	American
Frequency	1,018,444	305,298	145,464	101,984	82,009	71,014	66,626	60,222	56,753
Term	Covid	America	Obama	GOP	China	Black	Voter	Rally	World
Frequency	52,877	48,681	31,158	29,380	25,951	24,337	23,769	23,473	23,161
Phrase	Vote Trump	Trump Supporter	White House	Vote Biden	Supreme Court	Hunter Biden	Fox News	Kamala Harris	Melania Trump
Frequency	43,309	38,600	25,183	24,564	13,258	11,122	10,739	7822	7003
Phrase	Trump Impeachment	Trump Pardon	Trump Wall	Executive Order	Fake News	Tara Reade	Ivanka Trump	Black People	Election Fraud
Frequency	5965	5771	5349	5163	5052	4488	4483	4443	4378

shows the most frequent terms and phrases among these Tweets.

A rolling group of graduate and undergraduate students (with at least three students at a time) majoring in politics, journalism, communication, and international security along with a faculty in information sciences inspected and discussed 40,000 Tweets over a course of six months. Each student went through a minimum of 20 h of training before starting to label the Tweets. Weekly meetings were held to discuss students’ questions and address disagreements on labeling the Tweets.

The only non-random rules for selecting 40,000 Tweets, from the entire dataset were that an equal number of Tweets were selected from each day and not more than one Tweet was selected from the same source each day. When discussing Tweets, the first question is whether it is relevant to the U.S. 2020 presidential election. If not, the Tweet will be labeled as Irrelevant and not further inspected. If yes, the Tweet will be manually classified in three different ways: whether it contains disinformation, whether it contains extreme opinion, and whether it is rightist, leftist, or neutral. The Tweet will be labeled in accordance with the answer to these questions.

It is important to emphasize that we would only verify the factuality of the information in a Tweet, if it is in fact information, and not opinion. For instance, if a Tweet properly quotes a politician, but then provides the author’s opinion on the quote, while being clear that the second part is the author’s opinion, it will not be considered disinformation, regardless of what the author’s opinion is. To verify the factuality of quotes, we would primarily look for videos or audios and alternatively cross-check multiple valid and reputable news sources. If a quote is paraphrased, twisted, or taken out of context, in a way that it no longer matches the original message or conveys additional information that were not meant in the original quote, it will be considered disinformation. Many Tweets contain statistics about polls or Coronavirus, information about political appointees, bills that are being discussed or passed by the Congress, and foreign policies. The factuality of such information is verified by cross-checking valid and reputable news outlets. We define a valid and reputable news source as a news organization that has a history of presenting factual information, such as ABC, CBS, CNN, FOX, MSNBC, NBC, New York Times, USA Today, Wall Street Journal, and Washington Post. We rely, most often, on news sources that have little to no bias, such as Associated Press, Reuters, Axios, NPR, The Hill, and USA Today. If the information only appears in sources with perceived bias towards one political side, such as New York Times, Washington Post, Wall Street Journal, CNN, or FOX, we would take the bias into account in our assessment and would invest more effort into verifying its factuality. For instance, if a piece of information only comes up on right-leaning or left-leaning sources, we would question its validity, whereas if it comes up on both right- and left-leaning sources, NPR, or Axios, we would consider it as factual.

Additionally, the intent of each Tweet is identified. A Tweet might have more than one intent. Each Tweet is individually and independently investigated and discussed to identify its intent. For instance, after carefully studying the first Tweet, it might appear that its intent is to admire Republicans for their handling of the Coronavirus. Therefore, a class of intent with this title is defined and attributed to this Tweet. Instead of attempting to fit the following Tweets into existing classes of intent, their intent is independently identified. If such intent already exists among classes, it will be attributed to the Tweet, otherwise, a new class of intent is created. However, it is assured that there are no duplicate or overlapping classes of intent. There are no requirements on the minimum number of Tweets attributed to each class of intent. In other words, a class of intent could contain only one Tweet. There is also no upper limit on the number of classes and they are defined as needed.

After inspecting 75% of Tweets, there were 171 classes of intent defined. The intent of the remaining 25% of Tweets fit into the existing classes and no new classes were needed to be defined. This shows that the existing classes had reached a level of comprehensiveness that they were able to cover the intent of the remaining 25% of Tweets. The defined classes and their definitions were revised and adjusted multiple times throughout this study to best represent the intent of Tweets.

To assess the intercoder reliability and due to the subject’s nuanced nature, each Tweet is independently classified by two different people. In different cases, the two individuals disagreed on classification of 5% to 8% of their Tweets, with an average of 6.8% across the board. Such disagreements were eventually resolved by discussions among at least three people, including the two people who initially classified the Tweet.

3. Definitions

The following classes and intents are defined based on a study and multiple revisions of 40,000 Tweets.

3.1. Classes of Political Tweets

• Information: Tweets that contain true information, quotes, news, and news articles whose factuality can be verified through valid and reputable outlets.
• Disinformation: Tweets that create or propagate false information, false news, or lies, whose falsehood can be assessed through valid and reputable outlets. Tweets that create or promote unproven conspiracy theories.
• Opinion: Tweets that carry admiration or criticism of different matters, without spreading hateful or divisive ideologies or undermining the United States political system or federal and local organizations. The factuality of opinions cannot be assessed or is irrelevant because they state an individual’s opinion.
• Extreme Opinion: Tweets that aim to divide or polarize America based on political party, race, religion, etc., or undermine the U.S. political system or federal and local organizations by creating or promoting radical and violent ideologies and opinions. The factuality of opinions cannot be assessed or is irrelevant because they state an individual’s opinion.
• Leftist: Tweets that promote Liberals, Democrats, or their interests and agenda or demote Conservatives, Republicans, or their interests and agenda.
• Rightist: Tweets that promote Conservatives, Republicans, or their interests and agenda or demote Liberals, Democrats, or their interests and agenda.
• Neutral: Tweets without bias towards a political party, which do not promote or demote either rightist or leftist agenda and groups or equally criticize both Democrats and Republicans.

3.2. Intents of Political Tweets

1.

Admiring Trump and Republicans Handling of the Coronavirus Pandemic: Tweets that admire Trump and Republicans’ management and leadership of the Coronavirus pandemic, and his campaign’s and personal help and donations to hospitals and Coronavirus relief (3.88% of relevant Tweets).

2.

Criticizing Trump and Republicans Handling of the Coronavirus Pandemic: Tweets that criticize Trump and Republicans’ management and leadership of the Coronavirus pandemic, point out policy deficiencies, and ask Trump and Republicans to approve specific policies in this regard (25.43% of relevant Tweets).

3.

Admiring Democrats Handling of the Coronavirus Pandemic: Tweets that admire Democrats’ management and leadership of the Coronavirus pandemic (0.43% of relevant Tweets).

4.

Criticizing Democrats Handling of the Coronavirus Pandemic: Tweets that criticize Democrats’ management and leadership of the Coronavirus pandemic, point out policy deficiencies, and ask Democrats to approve specific policies in this regard (2.18% of relevant Tweets).

5.

Admiring CDC and Dr. Fauci’s Handling of the Coronavirus Pandemic (0.48% of relevant Tweets).

6.

Criticizing CDC and Dr. Fauci’s Handling of the Coronavirus Pandemic (0.40% of relevant Tweets).

7.

Blaming Obama for Mishandling of the Personal Protective Equipment (PPE) Storage: Tweets that blame the Obama administration for PPE shortage during the Coronavirus pandemic (0.08% of relevant Tweets).

8.

Criticizing China for Its Handling of the Coronavirus (1.15% of relevant Tweets).

9.

Downplaying the Severity, Fatalities, or Impacts of the Coronavirus: For instance, by assimilating it to the Flu virus (0.20% of relevant Tweets).

10.

Exaggerating the Severity, Fatalities, or Impacts of the Coronavirus (0.01% of relevant Tweets).

11.

Opposing Vaccination (0.04% of relevant Tweets).

12.

Supporting Vaccination (0.03% of relevant Tweets).

13.

Opposing Coronavirus Restrictions: Such as masks, canceling campaign rallies, postponing the opening of the economy and schools, etc. (0.56% of relevant Tweets).

14.

Supporting Coronavirus Restrictions (1.16% of relevant Tweets).

15.

Highlighting the Impact of the Coronavirus Pandemic on Economy and Employment (0.34% of relevant Tweets).

16.

Providing News or Results of Polls Regarding the Coronavirus Pandemic: Tweets that provide news or quotes from experts regarding the Coronavirus pandemic and its management and poll results on people’s opinion on the government’s handling of the pandemic (3.29% of relevant Tweets).

17.

Highlighting the Disproportionate Impact of the Coronavirus Pandemic Based on Race, Wealth, Gender, Age, etc. (0.04% of relevant Tweets).

18.

Offering Theories about the Origin or Cause of the Coronavirus (1.63% of relevant Tweets).

19.

Introducing Drugs or Treatments for the Coronavirus (1.08% of relevant Tweets).

20.

Highlighting that the Coronavirus Pandemic is Stealing Attention from the Election and Minimizing the Media’s Coverage of Biden (0.12% of relevant Tweets).

21.

Accusing Trump and Republicans of Racism, Supremacy, Sexism, Homophobia, and Transphobia: Tweets that point out that there are not enough Black Republicans (1.65% of relevant Tweets).

22.

Accusing Democrats of Racism, Supremacy, Sexism, Homophobia, and Transphobia (0.33% of relevant Tweets).

23.

Rejecting the Accusation that Republicans are Racist, Supremacist, Sexist, Homophobic, or Transphobic (0.19% of relevant Tweets).

24.

Rejecting the Accusation that Democrats are Racist, Supremacist, Sexist, Homophobic, or Transphobic (0.02% of relevant Tweets).

25.

Condemning or Accusing Republicans of Sexually Assaulting Women or Children: Tweets that condemn Republicans of not supporting the “Me Too” movement sufficiently (1.34% of relevant Tweets).

26.

Condemning or Accusing Democrats of Sexually Assaulting Women or Children: Tweets that condemn Democrats of not supporting the “Me Too” movement sufficiently (2.14% of relevant Tweets).

27.

Defending Republicans Against or Rejecting Accusations of Sexual Assault Toward Women or Children (0.08% of relevant Tweets).

28.

Defending Democrats Against or Rejecting Accusations of Sexual Assault Toward Women or Children (0.36% of relevant Tweets).

29.

Opposing Abortion Rights (0.22% of relevant Tweets).

30.

Supporting Abortion Rights (0.03% of relevant Tweets).

31.

Opposing LGBTQIA+ Rights (0.03% of relevant Tweets).

32.

Supporting LGBTQIA+ Rights (0.08% of relevant Tweets).

33.

Criticizing Gun Rights: Tweets that criticize the 2nd Amendment and ask for restrictions on gun rights (0.10% of relevant Tweets).

34.

Supporting Gun Rights: Tweets that defend the 2nd Amendment and criticize restrictions on gun rights (0.12% of relevant Tweets).

35.

Criticizing Free College Policies (0.01% of relevant Tweets).

36.

Supporting Free College Policies (0.01% of relevant Tweets).

37.

Criticizing Socialism and Fascism: and Tweets that portray Democrats as socialists and fascists (0.63% of relevant Tweets).

38.

Supporting Socialism and Fascism (0.03% of relevant Tweets).

39.

Concerning Climate Change: Tweets that endorse the factuality of climate change and express concern regarding its impacts (0.10% of relevant Tweets).

40.

Denying Climate Change: Tweets that deny the factuality of climate change (0.02% of relevant Tweets).

41.

Supporting Immigration and Opposing Nationalism: Tweets that criticize Trump and Republicans’ immigration policies, oppose nationalism, support globalism, immigration, birthright citizenship, and militarily or financial aid to other countries (0.51% of relevant Tweets).

42.

Opposing Immigration and Supporting Nationalism: Tweets that praise Trump and Republicans’ immigration policies, support nationalism and America First, oppose globalism, legal and illegal immigration, birthright citizenship, and militarily or financial aid to other countries (0.40% of relevant Tweets).

43.

Reporting Trump and Republicans’ Immigration Policies: Tweets that report Trump and Republicans’ Immigration policies in a neutral, nonpartisan way (0.15% of relevant Tweets).

44.

Spreading Deep State and QAnon Conspiracy Theory: At the core of QAnon is the conspiracy theory that all American presidents between John F. Kennedy and Donald Trump have been working with a cabal of globalist elites called The Cabal in order to undermine American democracy and advance their own nefarious agenda. The theory is more anti-elite than explicitly anti-Semitic. The Cabal seeks to destroy American freedom and subjugate the nation to the wills of a world government. The agenda also includes pedophilia, blood sacrifice, Satanism, and other attention-getting transgressions. QAnon is hoping that The Storm is coming, Donald Trump is secretly working in league with Robert Mueller to arrest Hillary Clinton, Barack Obama, and other members of the Deep State who are working to destroy our nation. Sealed indictments have already been filed, and arrests—followed by military trials, and possibly executions—are coming any day now [28] (0.78% of relevant Tweets).

45.

Denying Deep State and QAnon Conspiracy Theory (0.22% of relevant Tweets).

46.

Undermining Election Reliability: Tweets that impugn the reliability of the election system or its results or claim that there is or there will be large-scale voter fraud (0.29% of relevant Tweets).

47.

Undermining U.S. Constitution: for instance, by calling it bad or flawed (0.03% of relevant Tweets).

48.

Undermining U.S. Judicial System and FBI: for not being just (0.21% of relevant Tweets).

49.

Admiring U.S. Judicial System and FBI: for being just (0.00% of relevant Tweets).

50.

Criticizing Black Lives Matter Protests: or referring to them as rioters and criminals (1.40% of relevant Tweets).

51.

Admiring Black Lives Matter Protests (0.86% of relevant Tweets).

52.

Criticizing the Police: and condemning the police violence against black people (0.55% of relevant Tweets).

53.

Admiring the Police: for taking over a difficult and dangerous job (0.16% of relevant Tweets).

54.

Expressing Frustration Towards All Candidates: Tweets that express frustration and disappointment with candidates from both parties and imply that none is qualified (0.46% of relevant Tweets).

55.

Asking for Improvement in the Election System (0.02% of relevant Tweets).

56.

Suggesting that America is Run by Corrupt Corporations or Criminals: Tweets suggesting that the American government is corrupt and run by big corporations and businesses who only profit themselves (0.06% of relevant Tweets).

57.

Offending or Insulting the Rich: by calling them the privileged rich or phrases, such as “eat the rich” (0.05% of relevant Tweets).

58.

Pointing out the Corruption of Christian Preachers and Evangelicals: Tweets that criticize some top Christian preachers and Evangelicals for taking sides in politics and making hypocritical statements, misguiding their followers, and covering up Trump’s misconducts (0.05% of relevant Tweets).

59.

Criticizing America’s Military for Its Large Budget or Actions: or its wars with other countries (0.10% of relevant Tweets).

60.

Admiring America or its Military (0.11% of relevant Tweets).

61.

Accusing Main-Stream Media of Fake News (0.40% of relevant Tweets).

62.

Defending Main-Stream Media Against Accusations of Fake News (0.18% of relevant Tweets).

63.

Pointing out the Right-Wing Media’s Bias: Tweets that point out that the right-wing media is biased in favor of Trump and Republicans and against Democrats. For example, Fox News’ silence when it comes to Trump and Republicans’ misconduct or their propaganda regarding Coronavirus (0.67% of relevant Tweets).

64.

Pointing out the Left-Wing Media’s Bias: Tweets that point out that the left-wing media is biased in favor of Democrats and against Trump and Republicans. For example, MSNBC’s silence when it comes to Democrats’ misconduct (2.77% of relevant Tweets).

65.

Criticizing the Right-Wing Media for Not Enough Bias (0.06% of relevant Tweets).

66.

Criticizing the Left-Wing Media for Not Enough Bias (0.13% of relevant Tweets).

67.

Criticizing Media’s Too Much Coverage of Trump: claiming that they are unintentionally doing him a favor by making him more popular. Tweets that ask the media to limit their coverage of Trump, his briefings, and announcements (0.26% of relevant Tweets).

68.

Supporting Freedom of Speech on Social Media: Tweets that oppose any type of censorship on social media, such as Twitter, Facebook, and Instagram (0.35% of relevant Tweets).

69.

Opposing Freedom of Speech on Social Media: Tweets that support censorship on social media, such as Twitter, Facebook, and Instagram (0.19% of relevant Tweets).

70.

Condemning Mail-in Voting: and suggesting that it is not reliable and will lead to election fraud (0.43% of relevant Tweets).

71.

Supporting Mail-In Voting (0.23% of relevant Tweets).

72.

Highlighting or Condemning Russia’s Interference in the U.S. Election (0.14% of relevant Tweets).

73.

Defending U.S. Intelligence Backstabbers: Tweets that defend U.S. intelligence personnel who flipped against their agency (e.g., Edward Snowden and Julian Assange) and escaped to other countries (0.02% of relevant Tweets).

74.

Attacking U.S. Intelligence Backstabbers: Tweets that attack U.S. intelligence personnel who flipped against their agency (e.g., Edward Snowden and Julian Assange) and escaped to other countries (0.00% of relevant Tweets).

75.

Criticizing China’s Intellectual Property Theft: from the U.S. and other western countries (0.67% of relevant Tweets).

76.

Justifying or Denying China’s Intellectual Property Theft: from the U.S. and other western countries (0.07% of relevant Tweets).

77.

Stating Electability Factors: Tweets stating what factors would increase or decrease the electability or popularity of presidential candidates (1.63% of relevant Tweets).

78.

Reporting on Election Polls and Results: Tweets that report statistics about presidential primaries, general election, polls, and deferrals of election dates (1.22% of relevant Tweets).

79.

Reporting on Economy and Stock Market (0.41% of relevant Tweets).

80.

Reporting on U.S. Health Policies (0.03% of relevant Tweets).

81.

Reporting on U.S. Foreign Policies: Tweets that report Trump’s meetings and phone calls with foreign heads of states, his foreign policies, and statements in a neutral, nonpartisan way (0.50% of relevant Tweets).

82.

Admiring Trump’s Economic Performance and Policies: Tweets that admire Trump’s handling of the economy and its growth (0.62% of relevant Tweets).

83.

Admiring Democrats’ Economic Performance and Policies (0.08% of relevant Tweets).

84.

Criticizing Trump’s Economic Performance and Policies: Tweets that criticize Trump’s handling of the economy (0.95% of relevant Tweets).

85.

Criticizing Democrats’ Economic Performance and Policies: such as tax policies (0.16% of relevant Tweets).

86.

Admiring Trump and Republicans’ Foreign Policies: Tweets that admire Trump’s foreign policies and statements, for example his close relationship with Putin (0.63% of relevant Tweets).

87.

Admiring Democrats’ Foreign Policies: Tweets that admire Democrats’ foreign policies and statements, for example easing sanctions on Iran (0.03% of relevant Tweets).

88.

Criticizing Trump and Republicans’ Foreign Policies: Tweets that criticize Trump’s foreign policies and statements, for example threatening other countries with war (1.56% of relevant Tweets).

89.

Criticizing the Bond between Trump and Putin (0.67% of relevant Tweets).

90.

Criticizing Democrats’ Foreign Policies: Tweets that criticize Democrats’ foreign policies and statements, for example Hillary Clinton’s actions regarding Benghazi (0.19% of relevant Tweets).

91.

Reporting on US policies on Federal Land, Environment, Energy, and Oil Extraction: in a neutral, nonpartisan way (0.05% of relevant Tweets).

92.

Criticizing Trump and Republicans’ policies on Federal Land, Environment, Energy, and Oil Extraction: that would hurt the environment (0.40% of relevant Tweets).

93.

Criticizing Democrats’ policies on Federal Land, Environment, Energy, and Oil Extraction: for instance, placing limitations on oil extraction (0.04% of relevant Tweets).

94.

Admiring Trump and Republicans’ policies on Federal Land, Environment, Energy, and Oil Extraction (0.05% of relevant Tweets).

95.

Admiring Democrats’ policies on Federal Land, Environment, Energy, and Oil Extraction (0.05% of relevant Tweets).

96.

Criticizing Trump and Republicans’ Policies on Social Security (0.04% of relevant Tweets).

97.

Criticizing Trump and Republicans’ Healthcare Policies (0.44% of relevant Tweets).

98.

Criticizing Democrats’ Healthcare Policies: and Obamacare (0.18% of relevant Tweets).

99.

Admiring Trump and Republicans’ Healthcare Policies (0.08% of relevant Tweets).

100.

Admiring Democrats’ Healthcare Policies: and Obamacare (0.13% of relevant Tweets).

101.

Criticizing Trump and Republicans’ Policies on Narcotics and Smuggling (0.02% of relevant Tweets).

102.

Criticizing Democrats’ Policies on Narcotics and Smuggling (0.00% of relevant Tweets).

103.

Admiring Trump and Republicans’ Policies on Narcotics and Smuggling (0.02% of relevant Tweets).

104.

Admiring Democrats’ Policies on Narcotics and Smuggling (0.01% of relevant Tweets).

105.

Criticizing Trump and Republicans’ Policies on Prison System and Death Sentence (0.05% of relevant Tweets).

106.

Criticizing Democrats’ Policies on Prison System and Death Sentence (0.02% of relevant Tweets).

107.

Admiring Trump and Republicans’ Policies on Prison System and Death Sentence (0.00% of relevant Tweets).

108.

Admiring Democrats’ Policies on Prison System and Death Sentence (0.00% of relevant Tweets).

109.

Criticizing Obama and Democrats’ Housing Policies (0.05% of relevant Tweets).

110.

Portraying Trump as the Likely and Deserving Election Winner: Tweets that portray Trump as a good President who deserves to win again. Tweets implying that Trump is more likely to win than Biden. Tweets that predict a high voter turnout for Trump or a low voter turnout for Biden. Tweets that criticize Democrats for insulting Trump. Tweets that criticize other countries for criticizing President Trump (3.26% of relevant Tweets).

111.

Portraying Biden as the Likely and Deserving Election Winner: Tweets that portray Biden as a good candidate who deserves to win. Tweets implying that Biden is more likely to win than Trump. Tweets that predict a high voter turnout for Biden or a low voter turnout for Trump (1.70% of relevant Tweets).

112.

Criticizing Trump and Republicans Generally: Tweets that criticize Trump and Republicans in a general, nonspecific manner, sometimes by linking to other pages and videos (24.83% of relevant Tweets).

113.

Criticizing Biden and Democrats Generally: Tweets that criticize Democrats in a general, nonspecific manner, sometimes by linking to other pages and videos (8.15% of relevant Tweets).

114.

Appealing to Voters by Republicans: Tweets that encourage Republicans to come together and vote for their nominee, regardless of who it is. Tweets that encourage Republicans to vote for Trump. Tweets that advertise Trump’s campaign products (0.45% of relevant Tweets).

115.

Appealing to Voters by Democrats: Tweets that encourage Democrats to come together and vote for their nominee, regardless of who it is. Tweets that encourage Democrats to vote for Biden (0.94% of relevant Tweets).

116.

Appealing to Black Voters by Republicans: Tweets that portray Republican candidates as popular among blacks or appeal to blacks to vote for Republicans (0.07% of relevant Tweets).

117.

Appealing to Black Voters by Democrats: Tweets that portray Democratic candidates as popular among blacks or appeal to blacks to vote for Democrats (0.03% of relevant Tweets).

118.

Fact-Checking Trump and Republicans and Pointing Out Their Lies: Tweets that fact-check Trump’s statements and bring attention to his lies. Tweets that highlight the importance of fact-checking Trump. Tweets that admire the media for fact-checking Trump. Tweets that demand Trump to tell the truth (2.16% of relevant Tweets).

119.

Fact-Checking Democrats and Pointing Out Their Lies: Tweets that fact-check Democrats’ statements and bring attention to their lies. Tweets that highlight the importance of fact-checking Democrats. Tweets that admire the media for fact-checking Democrats. Tweets that demand Democrats to tell the truth (0.32% of relevant Tweets).

120.

Pointing out Trump’s Foolish Statements and Temper-Tantrums: Tweets that bring attention to Trump’s foolish statements and his temper-tantrums when faced with criticism (1.70% of relevant Tweets).

121.

Pointing out Biden’s Foolish Statements and Temper-Tantrums: Tweets that bring attention to Biden’s foolish statements and his temper-tantrums when faced with criticism (0.08% of relevant Tweets).

122.

Pointing out Trump’s Hypocrisy: Tweets that directly or indirectly (sarcastically) point out the hypocrisy or contradiction in some of Trump’s statements or actions (0.60% of relevant Tweets).

123.

Pointing out Republicans’ Hypocrisy: Tweets that directly or indirectly (sarcastically) point out the hypocrisy or contradiction in some of Republicans’ statements or actions (0.47% of relevant Tweets).

124.

Pointing out Democrats’ Hypocrisy: Tweets that directly or indirectly (sarcastically) point out the hypocrisy or contradiction in some of Democrats’ statements or actions (0.91% of relevant Tweets).

125.

Pointing out Trump and Republicans’ Corruption: Tweets that point out how Trump, top Republicans, and their families receive special treatment from the government and judicial system. For example, Tweets that point out that people in Trump’s administration work in his campaign too (which is not legal) and Tweets that reveal the deals that the government is making with or to the benefit of Trump’s businesses (2.63% of relevant Tweets).

126.

Pointing out Biden and Democrats’ Corruption: Tweets that point out how top Democrats and their families receive special treatment from the government and judicial system (1.12% of relevant Tweets).

127.

Suggesting that Rich People Support Trump and Republicans (0.11% of relevant Tweets).

128.

Suggesting that Rich People Support Biden and Democrats (0.06% of relevant Tweets).

129.

Criticizing Trump’s Policies and Statements by Other Republicans: Tweets where Republicans criticize Trump’s policies or statements (0.26% of relevant Tweets).

130.

Insulting or Threatening Trump and Republicans: Tweets that insult, offend, wish death, or suggest to kill Trump and Republicans. Tweets that exaggerate in portraying Trump and Republicans as racist. Tweets that accuse Trump and Republicans of being murderers because of their mismanagement of the Coronavirus pandemic (2.53% of relevant Tweets).

131.

Insulting or Threatening Democrats: Tweets that insult, offend, wish death, or suggest to kill Democrats. Tweets that accuse Democrats of being murderers because of their mismanagement of the Coronavirus pandemic (0.74% of relevant Tweets).

132.

Praising Trump’s Bond with Christianity: Tweets that portray Trump as a strong Christian and admire his Christianity. Tweets that display Trump as pro-life and anti-abortion (0.19% of relevant Tweets).

133.

Criticizing Trump’s Bond with Christianity: Tweets that criticize Trump’s bond with Christianity and his Christian supporters, mostly because it stands against science, abortion rights, or LGBTQIA+ rights (0.05% of relevant Tweets).

134.

Praising Republicans for Being Christian: Tweets that praise Republicans for abiding by Christian rules and standing against science, abortion rights, or LGBTQIA+ rights (0.02% of relevant Tweets).

135.

Criticizing Republicans for Being Extreme Christian: Tweets that criticize Republicans for being extreme Christian, mostly because it stands against science, abortion rights, or LGBTQIA+ rights (0.05% of relevant Tweets).

136.

Praising Democrats for not being Christian: Tweets that praise Democrats for not abiding by Christian rules and standing for science, abortion rights, or LGBTQIA+ rights (0.00% of relevant Tweets).

137.

Criticizing Democrats for not being Christian: Tweets that portray Democrats as not Christian or pro other religions (0.10% of relevant Tweets).

138.

Questioning the Physical and Mental Health of Democratic Candidates: Tweets that portray Democratic candidates or politicians as unhealthy or claim that they have health issues that prevent them from properly performing their tasks if they get elected to the office (0.86% of relevant Tweets).

139.

Questioning the Physical and Mental Health of Republican Candidates: Tweets that portray Republican candidates or politicians as unhealthy or claim that they have health issues that prevent them from properly performing their tasks if they get elected to the office (0.64% of relevant Tweets).

140.

Praising Trump’s Physical and Mental Health: Tweets that portray Trump as a strong and healthy person (0.03% of relevant Tweets).

141.

Praising Biden’s Physical and Mental Health: Tweets that portray Biden as a strong and healthy person (0.01% of relevant Tweets).

142.

Dividing Republicans: Tweets that attempt to sow division among Republicans based on who they supported during the primaries or other reasons (0.11% of relevant Tweets).

143.

Dividing Democrats: Tweets that attempt to sow division among Democrats based on who they supported during the primaries or other reasons (0.85% of relevant Tweets).

144.

Praising Violence Against Critics of Trump or His Policies: Tweets that praise or invite to violence against Trump’s critics or praise Trump’s and his supporters’ violence towards Trump’s critics. Tweets that portray Trump as a powerful president because of his violence towards his opponents, for example using fictional video (0.15% of relevant Tweets).

145.

Condemning Violence Against Critics of Trump or His Policies: Tweets that condemn violence against Trump’s critics or condemn Trump’s and his supporters’ violence towards Trump’s critics. Tweets that portray Trump as a violent or aggressive president because of his violence towards his opponents (0.34% of relevant Tweets).

146.

Interpreting the Results of Mueller Investigation in Favor of Republicans (0.20% of relevant Tweets).

147.

Interpreting the Results of Mueller Investigation in Favor of Democrats (0.01% of relevant Tweets).

148.

Reporting the Results of Mueller Investigation: in a neutral and non-partisan way (0.04% of relevant Tweets).

149.

Opposing Trump’s Impeachment: Tweets claiming that the initiation and conduction of Trump’s impeachment were unjust, unjustified, and unfair (0.21% of relevant Tweets).

150.

Supporting Trump’s Impeachment: Tweets claiming that the initiation and conduction of Trump’s impeachment were just, justified, and fair (0.30% of relevant Tweets).

151.

Criticizing Trump for Too Many Firings: Tweets highlighting that Trump fires too many people because of their opposition to his policies or criticism of his decisions or statements (2.32% of relevant Tweets).

152.

Criticizing Trump’s Pardons: Tweets that criticize Trump for pardoning too many people or pardoning individuals who are related to him (0.03% of relevant Tweets).

153.

Supporting Trump’s Pardons (0.03% of relevant Tweets).

154.

Criticizing Trump’s Undermining and Domination of Federal Agencies and Organizations: Tweets that criticize Trump for his gutting of Federal agencies and organizations (0.12% of relevant Tweets).

155.

Criticizing Trump’s Overuse of Social Media: such as Twitter (0.69% of relevant Tweets).

156.

Criticizing Trump’s Response to Natural Disasters, Other Than Coronavirus: such as hurricanes (0.03% of relevant Tweets).

157.

Criticizing Trump’s Choice of Appointees and Judges (0.22% of relevant Tweets).

158.

Admiring Trump’s Choice of Appointees and Judges (0.09% of relevant Tweets).

159.

Reporting Trump’s Choice of Appointees and Judges: in a neutral and non-partisan way (0.13% of relevant Tweets).

160.

Criticizing Republican Politicians for Not Standing up Against Trump’s Policies and Statements: when they are wrong (0.24% of relevant Tweets).

161.

Blaming Trump’s Supporters for Their Blind Support: regardless of his mishandling of various issues (1.92% of relevant Tweets).

162.

Downplaying Trump’s Wealth: Tweets that question Trump’s wealth, stating that Trump’s net worth is less than what Trump claims (0.44% of relevant Tweets).

163.

Claiming That There Are Information Leakers in Trump’s Administration: Tweets implying that there are people in Trump’s administration who leak information out to journalists or the public (0.09% of relevant Tweets).

164.

Calling out Republicans on Their Voter Suppression: Tweets that call out Republicans on their strategies to suppress Democratic voters. For example, Tweets that condemn Republicans for their opposition to mail-in voting, despite the Coronavirus pandemic (0.69% of relevant Tweets).

165.

Debating Democratic Primaries: Tweets that discuss and compare qualifications of different Democratic candidates with each other (0.35% of relevant Tweets).

166.

Criticizing Democrats for Their Humiliation of America: Tweets that criticize mocking of America or calling America divided by its own people or foreigners, for example because of its healthcare system (0.06% of relevant Tweets).

167.

Linking Democrats to Terrorist Groups: Tweets that spread propaganda and conspiracy theories regarding links between Democrats (e.g., Obama) and terrorist groups (0.35% of relevant Tweets).

168.

Delegitimizing Any Investigation of Trump and Republicans: Tweets that portray any investigation of Trump and Republicans as baseless and a witch hunt by Democrats (0.68% of relevant Tweets).

169.

Delegitimizing Any Investigation of Democrats: Tweets that portray any investigation of Democrats as baseless and a witch hunt by Republicans (0.06% of relevant Tweets).

170.

Implying that FBI, Federal Courts, and Judges are Biased Against Trump and Criticizing Them by Trump and Republicans: for instance, Tweets that criticize supreme court Justices’ decisions, when they are not in favor of Trump or his policies (0.24% of relevant Tweets).

171.

Taking the Blame off of Trump on Different Issues by Downplaying the Presidential Powers and Authorities: Tweets claiming that the federal government has no or limited power and authority on various issues and only the states make the decisions, thus Trump is not to blame (0.07% of relevant Tweets).

4. Example Tweets

Table 2. Example Tweets and their classes: I stands for information, D for disinformation, O for opinion, E for extreme opinion, R for rightist, N for neutral, and L for leftist.

Tweet	I	D	O	E	R	N	L	Intent Classes
Still waiting for someone to show me one example of Trump being racist that isn’t fake news.			✓		✓			23, 61, 64
Democrats panic as Trump polls 10 points higher than election day 2016! #Democrats will reap what they have sown. The segregating, war mongering, divisive party that founded the KKK and are now under communist & extremist control is finished. #RipDems		✓		✓	✓			37, 78, 110, 131
@Trump shows no urgency as #Americans die & #Covid19 damages the U.S. #Economy! But #lie as he will, #Mueller showed #Trump was #guilty of #indictable #RussianCollusion! and all that saved him was that under #JusticeDeptPolicy, a seated U.S. #President can’t be #indicted!		✓	✓				✓	2, 82, 143
Leaked border patrol memo tells agents to send migrants back immediately—ignoring asylum law—propublica Trump and his border. They could drop a nuke on us and this fool would still be crying about the border!	✓		✓				✓	41, 112
If you are made to believe that the main stream media is #FakeNews then all you will have left to trust for news is Trump, social media and Fox News. This is why Trump spends every day attacking the great journalists working so hard to report the truth. #FreedomOfPress.			✓				✓	62, 63
I agree with Trump, mail in voting can be adjusted, tampered and manipulated; in short, it’s a process that attracts fraud. In my state, since adoption, it seems the Democratic candidates have won the close races. I miss the days of going to the polls, showing my id and voting.		✓	✓		✓			46, 70
I’ll repeat, Trump hired Kris Kobach in 2017 to investigate voter fraud, spending millions of taxpayer dollars & again the Republicans found nothing! I’m sick of their lies about voter id because they know if they can’t suppress the vote they’ll lose #TrumpPressBriefing #VoteBlue	✓		✓				✓	164
Prosecutors using the federal sentencing guidelines recommend an “appropriate” sentence for Roger Stone. One Trump tweet later, 4 career attorneys resign as DOJ takes the unusual step to reverse their recommendation. Why?	✓						✓	125
Democrats: America is divided! Me: because you divided America by calling literally everyone who isn’t a Democrat “deplorables and racist.” Democrats: we must come together! Me: if you actually wanted people to come together then you’d stop lying and tell them how to do it.			✓		✓			113, 119
The differences in your choices: choose Trump and he will give you the ability to get what you want out of life! Choose Democrats: they will give you a life completely controlled by big government, you get and do what they tell you! Your choice, choose wisely!			✓		✓			37, 114
Despite announcing he would force GM to make ventilators, Trump has not ordered any.	✓						✓	2, 122
President Trump requests absentee ballot. Can we say hypocrite?	✓		✓				✓	122
So UBI, allowing immigrants to actually work here legally is okay when Republicans like your president say it is, but when other parties say it, it’s considered “socialism”. Pandemic really showing your guys true colors.			✓				✓	123
Ivanka Trump’s Chinese trademarks raise corruption concerns.	✓						✓	125
@JoeBiden as Trump ramps up attacks on your son, it’s time to counter with the illegal deals Kushner made with Deutsche Bank and Don Junior’s shady deals. Also, Kushner blocked Qatar from global meetings since they refused to loan Kushner $. #TrumpKushnerDeutscheBank	✓						✓	125
@DanHenninger What is happening with the Biden family Ukraine, China, Iraq Investigations? Is not the Ukraine doing their own Investigation? Is Biden getting the Criminal Candidate Exemption from prosecution that Hillary Clinton got?		✓	✓		✓			126
King Jehoshaphat & Donald J Trump seem to share the same emotions concerning disasters. Both are calling upon the people to pray & worship God. We should do the same. (2 Chron. 20: 9)			✓		✓			1, 132
@MaisonDejenn I’ve sat quietly and watched your tweets and retweets. I get it. You like Bernie. You wanna keep driving folks to not voting? Which means Trump. You want this?			✓				✓	165
FBI may have obtained FISA warrants with nonexistent supporting docs. Huge Obama/Biden scandal, both must answer and many including them, Comey Rosenstein, Yates, McCabe must be prosecuted-This scandal w all of the spying done throughout Obama presidency.		✓		✓	✓			48, 146
Nancy Pelosi admits that Democrats “knew about” the forthcoming Coronavirus outbreak, but chose to pursue their partisan impeachment regardless.		✓			✓			4, 149
Editorial: Trump wields EPA to help, not regulate, oil and gas.			✓				✓	92, 154
Almost 8 h without any #Trump tweets! Maybe his internet is down.			✓				✓	155
Hi, just a reminder that while millions of people are dying due to, oh yeah, the Coronavirus, Trump decides it’s time to talk about how he’s the number 1 most popular person on Facebook.	✓						✓	2, 120, 155
‘This is despicable’: not even Covid-19 pandemic can halt Trump’s right-wing takeover of federal courts.	✓						✓	157
#TrumpIsBroke The Trump organization is looking for a Coronavirus bailout.	✓						✓	162
Trump and the RussiaPublicans are saying uninsured Americans will be reimbursed for the cost of Covid-19 treatment. I am sorry but Trump guaranteeing he will pay anyone is a joke, the bankruptcy king!	✓		✓				✓	2, 97, 125, 162
Trump’s opposition to absentee voting is voter suppression. Plain and simple.			✓				✓	164
Trump would be awesome for education. Problem is only the states write the rules on it.		✓			✓			110, 171
The secret backstory of how Obama let Hezbollah off the hook #Democrats #Obama #Politics		✓			✓			167
This is correct. If people are allowed to vote Dems Win. This Quote is from Georgia. If they know it is game over there, how many other Red States really aren’t? Georgia GOP Leader: more absentee voting will help turnout, be ‘devastating to Republicans’	✓		✓				✓	71, 164
Sure. Trump “loses confidence” in the Inspector General of the intelligence community for doing his legal and procedurally prescribed job. Trump got caught being a corrupt and lying crook, so he fires the IG. In the middle of a national crisis. #ImpeachedTrump	✓		✓				✓	125, 150, 151
President Trump in a national emergency you can shut down any media and or government officials speaking against the national interest whose treasonous acts would be unacceptable.				✓	✓			144
Has anyone else noticed that Biden’s dementia has gotten a lot worse ever since the last debate where he was surprisingly coherent? I wonder if he took an amphetamine for the debate, and the negative side effects of a dementia patient doing this is why he has gotten so much worse.		✓			✓			138
If you believe in God, you are a Republican, because Democrats, Demonrats believe is Satan. If you are a Democrat, forget about God, because he has forsaken you and allowed Coronavirus to feast on you first. Even now the red countries are mostly safe from infection.				✓	✓			131, 134, 137
Romney warns Trump: do not interfere with Coronavirus relief oversight. Thanks for trying to pretend that we still have some semblance of a democracy @SenatorRomney	✓		✓				✓	129
Are y’all ready for the wave of celebrities who will openly support Biden when he wins the nomination?			✓				✓	128
I said FBI is under Obama’s command. Trump needs to investigate Obama’s laws with FBI and IRS and Hunter Biden in China and other countries. I think Democrats are behind this virus because you were doing a great job and Democrats hated this.		✓			✓			18, 44, 126
Michigan governor threatened doctors who used promising Covid-19 drug, now she’s begging Trump for it.	✓				✓			4, 19, 124
All of the feminists taking #MeToo out of their bios and deleting their past tweets about believing women are tacitly admitting they think Biden is guilty but they do not want to be forced to say it out loud.	✓		✓		✓			26, 124
Joe Biden gets fact checked over claim that Trump admin did not act early enough.	✓				✓			1, 119
MSNBC dumps Trump Coronavirus briefing to do live fact check of lies #USRC	✓						✓	118
Do you want to be on the Democrats plantation or do you want to be free and have less government in your lives! Vote for a free America! “America 2.0”				✓	✓			22, 37, 114
To me it feels like the Republicans are the parents, and the Left are the whiny kids that need a good ass woopin.			✓		✓			113
Over the years I’ve slowly realized my stepdad is Donald Trump. Racist, married one of the nationalities he makes fun of, homophobic, pretending to know about everything while knowing nothing. Likes the word “yuge” and often times says things… then forgets they were said.			✓				✓	21, 112, 139
During this catastrophe the American people desperately need what Trump is uniquely incapable of delivering… The truth, leadership, compassion & empathy, trust, the ability to unite. Joe Biden has all that to offer and more and he’s already proven himself. #TeamJoe #Covid19			✓				✓	2, 111, 112, 118
Donald Trump is doing exactly what Putin wanted him to, systematically dismantle America. Hillary Clinton warned you he was a puppet.			✓				✓	89, 112
Former CIA director scolds Trump for calling Saudi Crown Prince ‘friend’ exactly 18 months after Khashoggi murder.	✓						✓	88
Democrats seeing opportunity to decapitate the United States capitalist economy. They are salivating over it. For them it’s the crisis opportunity of a lifetime to push through agenda. Despite facts show that it is capitalism and farmers, frackers, and truckers who are saving us.				✓	✓			37, 85
President Trump, a man of action has praised Xi of Communist China, while at the same time signing a budget-busting unprecedented socialist bailout that will put the nail in the economic coffin of this country- WND @MarkLevinShow @GrahamLedger @AnnCoulter	✓		✓			✓		37, 84, 88
Free read: oil prices record biggest ever, single day of gains, after Trump raises hopes of end to Saudi-Russia #OilPriceWar. Also, US #NatGas benchmark Henry Hub plunges to lowest price this millennium.	✓				✓			82, 86
Trump’s reelection hopes may depend on blue America’s rebound. I’m torn.	✓						✓	15, 77
#US officials agree on new ways to control high tech exports to #China–sources. The Trump administration is tightening rules to prevent China from obtaining advanced U.S. technology for commercial purposes and then diverting it to military use.	✓					✓		75, 86
Hey dipshit asshole Democrats no voting from home. You liars & criminals are always trying to steal elections, it is the only way you can “win” after all.				✓	✓			70, 113, 131
The pro-Trump fake news media won’t stop lying even when the world can see that they are! This time they are definitely anti-national as they are causing thousands of deaths by Coronavirus! It’s you who are a hoax! You are the worst journalists the world has ever seen!			✓				✓	2, 61, 63
Anti-science Christians who went ‘all in’ for Trump bear responsibility for Covid-19 crisis: religious extremism expert #SmartNews	✓						✓	2, 135
QANON, a baseless conspiracy theory boosted by Trump, continues creeping into mainstream politics.	✓						✓	45
Q-the plan to save the world remastered. Wake up Americans, it’s going to happen. Americans need to stop putting criminals in office, Trump will do the rest & the few that we know are for America & you know who they are. It’s a new beginning!		✓			✓			44
@realDonaldTrump @WhiteHouse @VP @GOP Every day now America is starting to feel like 1938 Germany with a disarmed population because of Liberal left-wing Democrats. Make universal concealed carry the law of the entire USA.				✓	✓			34, 113
Anti-Asian incidents in United States appear to be spiking as @realDonaldTrump and @SecPompeo promoted their “Chinese virus” and “Wuhan virus” rhetoric.	✓						✓	21
@LindseyGrahamSC This virus did not start at any wet market! It was engineered in the lab in Wuhan China. Purposely released to cause the death of people and bring down president Trump’s economy! Wake up.		✓			✓			8, 18
@chipper484 Coronavirus-A bio weapon created for population control, control of world markets, world economies, world politics. Democrats created a select committee to stop Coronavirus task force, they’re sacrificing us.		✓			✓			18
The past 2 weeks wiped out all the economy’s job gains since president Trump’s November 2016 election, a sign of how rapid, deep and painful the economic shutdown has been on American families.	✓						✓	15
This cure is so much worse than the virus. Is the devastation of millions of lives and families really worth saving the old sick and weak? Live in the hell being created or let the chips fall where they may. #Coronavirus #Business #FlattenTheCurve #USA Trump #StayHomeSaveLives				✓	✓			13
These Coronavirus commercials are starting to annoy the shit out of me, where were the H1N1 commercials? More people died from that… where was the shut down and outcry then? Oh wait, it is an election year and Trump is your president.			✓		✓			9, 13
Obama & the Democrats are to blame for the total unpreparedness of this country for the onslaught of the Coronavirus. They cut the health agency budgets to spend on giveaway programs gutting any plans for a national health emergency. Dems must go in Nov.		✓	✓		✓			4, 7, 114
President Trump needs to get rid of Doomsday Fauci. None of his predictions ever come true. Not even close. He doesn’t give a damn if the economy is shut down and destroyed, forever. Recommends not using drugs other doctors are having huge successes with. Can’t stand him!		✓	✓		✓			6, 13, 19
Deranged Trump supporters are sending so many death threats to Dr. Fauci—just because he dared to correct Trump’s barrage of lies about Coronavirus—that the Secret Service had to step up his security. Thank you for standing up for the truth, Dr. Fauci! #TrumpMadness	✓		✓				✓	2, 5, 118, 145
Democrats are creating a committee to investigate Trump’s response… in the middle of the response. Either they do not think this crisis is serious or they do not care about death. Can you imagine being in the middle of WWII & opening an investigation into FDR’s response to pearl harbor?	✓		✓		✓			4, 168
@MitchellReports Pelosi calls Trump a racist 31 Jan & urged “everyone should go visit Chinatown” that’s over 3 weeks after Trump calls the virus serious in State of Union speech closed China flights. DeBlasio told people go to concerts 6 weeks after!		✓			✓			1, 4

shows example Tweets from different classes. For instance, the second Tweet in this table is “Democrats panic as Trump polls 10 points higher than election day 2016! Democrats will reap what they have sown. The segregating, war mongering, divisive party that founded the KKK and are now under communist and extremist control is finished.” This is a right-leaning Tweet as it attacks the left. We labeled this Tweet as disinformation because we did not find any credible sources that Trump polls 10 points higher than the election day in 2016. There are also multiple unbiased and valid news sources that reject the claim that the Democratic Party founded KKK. We labeled this Tweet as extremist because it uses extreme language to describe the Democratic party. Four intents are attributed to this Tweet: (a) reporting on election polls, because it mentions that Trump polls 10 points higher than the election day in 2016; (b) portraying Trump as the likely election winner, because it implies that Trump is likely to win; (c) insulting Democrats; and (d) criticizing socialism and fascism, because at the end, the author expresses their dissent for communism.

As another example, the third Tweet in this table is “Trump shows no urgency as Americans die and Covid19 damages the U.S. economy! But, lie as he will, Mueller showed that Trump was guilty of indictable Russian Collusion and all that saved him was that under Justice Department Policy, a seated U.S. President can’t be indicted!”. This is a left-leaning Tweet as it criticizes Trump. It is disinformation because, according to the results of the Muller investigation, Muller did not show that Trump was guilty or indictable of collusion with Russians. It is disinformation because the author is attributing this conclusion to Muller and not as their subjective opinion or interpretation. It is labeled as Opinion, because the author is expressing his opinion that Trump is showing no urgency with regard to the Coronavirus pandemic and its impacts. It is not extremist because it is not propagating any violent or extreme ideas. Three intents are attributed to this Tweet: (a) criticizing Trump’s handling of the Coronavirus pandemic, because the author believes that Trump does not show any urgency in this regard; (b) criticizing Trump’s economic policies, because the author believes that the economy is damaged by the pandemic and Trump is not conducting the right policies to contain the damage; and (c) interpreting the results of Muller investigation in favor of Democrats, because the author claims that Muller showed that Trump colluded with Russians during the election.

5. Analytics and Discussions

Out of 40,000 Tweets, 93.73% were relevant to the U.S. 2020 presidential election.

Table 3. Percentage of relevant Tweets that contain disinformation or extreme opinions, with different political affiliations; R stands for rightist, N for neutral, L for leftist, D for disinformation, and E for extreme opinion.

R	N	L	R D	N D	L D	R E	N E	L E
25.64%	10.52%	63.86%	3.13%	0.13%	2.03%	2.04%	0.12%	1.79%

provides the percentage of relevant Tweets that contain disinformation or extreme opinions, with different political affiliations. Despite leftist Tweets outnumbering rightist Tweets by 2.5 times, rightist disinformation Tweets outnumber leftist disinformation Tweets by 1.54 times, and rightist extreme opinion Tweets outnumber leftist extreme opinion Tweets by 1.14 times. In other words, while 3.18% of leftist Tweets are disinformation and 2.80% of leftist Tweets are extreme opinions, 12.21% of rightist Tweets are disinformation and 7.96% of rightist Tweets are extreme opinions. This means that disinformation and extreme opinions are 3.8 and 2.8 times more prevalent among rightist Tweets than leftist Tweets.

Table 4. The most frequent terms and phrases among information, disinformation, opinion, extreme opinion, rightist, neutral, and leftist Tweets.

Information Tweets	Term	Coronavirus	Mask	China	TikTok	Ban	Crisis	Election	Death
	Frequency	18.26%	5.72%	3.46%	2.70%	2.50%	2.36%	2.18%	1.99%
Disinformation Tweets	Term	Coronavirus	China	Obama	Election	Death	Mueller	Anonymous	Pedophile
	Frequency	6.37%	5.18%	3.98%	3.58%	3.58%	3.50%	3.34%	3.18%
Opinion Tweets	Term	Coronavirus	Vote	Mask	Election	TikTok	Hate	China	Obama
	Frequency	6.04%	4.01%	2.91%	2.74%	2.48%	2.01%	1.97%	1.96%
Extreme Tweets	Term	Racist	Pedophile	Kill	Rapist	Death	Coronavirus	Black	Vote
	Frequency	5.76%	4.27%	4.27%	4.06%	3.84%	3.74%	3.74%	3.63%
Rightist Tweets	Term	Coronavirus	Pelosi	China	Vote	Mask	Obama	Election	FoxNews
	Frequency	10.36%	3.96%	3.91%	3.63%	3.17%	2.96%	2.48%	2.35%
Neutral Tweets	Term	Coronavirus	China	TikTok	Ban	Vote	Election	Lab	Mask
	Frequency	16.23%	6.49%	5.97%	5.01%	3.45%	3.21%	3.21%	3.01%
Leftist Tweets	Term	Coronavirus	Mask	Vote	Election	Death	TikTok	Crisis	Job
	Frequency	11.47%	4.57%	2.44%	2.32%	2.22%	1.93%	1.89%	1.74%

shows the most frequent terms among information, disinformation, opinion, extreme opinion, rightist, neutral, and leftist Tweets.

Table 5. Percentage of disinformation and extreme opinion Tweets falling in each class of intent, described in Section 3; R stands for rightist, N for neutral, L for leftist, D for disinformation, and E for extreme opinion.

L D		L E		R D		R E
Intent	%	Intent	%	Intent	%	Intent	%
112	29.67	130	38.92	113	24.73	113	32.23
25	28.01	112	30.90	4	11.83	44	17.36
2	20.54	25	25.71	126	9.27	131	16.74
130	8.30	2	19.81	64	9.14	50	10.74
125	6.43	161	11.08	44	8.20	4	10.12
21	5.81	21	10.38	1	7.80	110	8.26
161	5.81	52	4.01	50	7.12	126	6.40
84	2.70	145	2.59	26	6.18	37	5.58
89	2.07	51	1.65	146	5.65	64	4.55
120	1.87	37	1.42	131	4.84	167	4.55

provides the most common intents among disinformation and extreme opinion Tweets along with their percentages. The most common disinformation and extreme opinion intents among leftist Tweets are insulting or threatening Trump and Republicans, criticizing Trump and Republicans generally based on false premises, falsely accusing Republicans of mistreating women or children in a sexual way, and criticizing Trump and Republicans handling of the Coronavirus pandemic based on false premises.

The vast majority of rightist disinformation and extreme opinions are focused on criticizing Biden and Democrats generally based on false premises, criticizing Democrats handling of the Coronavirus pandemic based on false premises, falsely pointing out Biden and Democrats’ corruption, spreading deep state and QAnon conspiracy theories, insulting or threatening Democrats, accusing left-wing media of bias for false reasons, and criticizing Black Lives Matter protests in an extreme way.

Table 6. Percentage of relevant Tweets falling in each class of intent, described in Section 3.

Intent	%	Intent	%	Intent	%	Intent	%	Intent	%	Intent	%	Intent	%	Intent	%
2	25.43	78	1.22	82	0.62	167	0.35	90	0.19	163	0.09	133	0.05	24	0.02
112	24.83	14	1.16	122	0.60	145	0.34	98	0.18	7	0.08	135	0.05	73	0.02
113	8.15	8	1.15	13	0.56	15	0.34	62	0.18	32	0.08	11	0.04	101	0.02
1	3.88	126	1.12	52	0.55	22	0.33	53	0.16	83	0.08	17	0.04	103	0.02
16	3.29	19	1.08	41	0.51	119	0.32	85	0.16	99	0.08	93	0.04	134	0.02
110	3.26	84	0.95	81	0.50	150	0.30	43	0.15	27	0.08	96	0.04	10	0.01
64	2.77	115	0.94	5	0.48	46	0.29	144	0.15	121	0.08	148	0.04	36	0.01
125	2.63	124	0.91	123	0.47	129	0.26	72	0.14	76	0.07	30	0.03	141	0.01
130	2.53	51	0.86	54	0.46	67	0.26	66	0.13	116	0.07	47	0.03	147	0.01
151	2.32	138	0.86	114	0.45	170	0.24	159	0.13	171	0.07	87	0.03	35	0.01
4	2.18	143	0.85	97	0.44	160	0.24	100	0.13	56	0.06	156	0.03	104	0.01
118	2.16	44	0.78	162	0.44	71	0.23	154	0.12	166	0.06	38	0.03	49	0.00
26	2.14	131	0.74	3	0.43	29	0.22	20	0.12	65	0.06	140	0.03	74	0.00
161	1.92	164	0.69	70	0.43	45	0.22	34	0.12	128	0.06	153	0.03	107	0.00
120	1.70	155	0.69	79	0.41	157	0.22	127	0.11	169	0.06	12	0.03	102	0.00
111	1.70	168	0.68	92	0.40	48	0.21	60	0.11	58	0.05	31	0.03	108	0.00
21	1.65	63	0.67	6	0.40	149	0.21	142	0.11	94	0.05	80	0.03	136	0.00
18	1.63	89	0.67	42	0.40	146	0.20	33	0.10	105	0.05	117	0.03
77	1.63	75	0.67	61	0.40	9	0.20	59	0.10	109	0.05	152	0.03
88	1.56	139	0.64	28	0.36	23	0.19	137	0.10	57	0.05	40	0.02
50	1.40	86	0.63	68	0.35	69	0.19	39	0.10	91	0.05	55	0.02
25	1.34	37	0.63	165	0.35	132	0.19	158	0.09	95	0.05	106	0.02

provides the percentage of relevant Tweets, falling in each class of intent. Intents are ordered in this table based on their size among Tweets. A total of 31.43% of relevant Tweets are attributed to more than one class, while the remaining are attributed to only one. Unsurprisingly, Criticizing Trump and Republicans’ Handling of the Coronavirus Pandemic, is the most common intent among Tweets, where 25.43% of Tweets carry this intent. This highlights the dissatisfaction of Twitter users with Trump administration’s handling of the Coronavirus spread and its considerable impact on people’s vote. The second most common intent is criticizing Trump and Republicans generally, where 24.83% of Tweets inexplicitly express their dislike of Trump and other Republicans. The third most common intent is criticizing Biden and Democrats generally, where 8.15% of Tweets express their inexplicit dislike for Democratic politicians and candidates. The fourth most common intent is admiring Trump and Republicans’ handling of the Coronavirus pandemic, where 3.88% of Tweets express their admiration for how Trump and other Republicans have managed the Coronavirus pandemic and mitigated its impact on economy and employment.

The most common intents are providing news or results of polls regarding the Coronavirus pandemic (3.29%), portraying Trump as the likely and deserving election winner (3.26%), pointing out the left-wing media’s bias in favor of Democrats (2.77%), pointing out Trump and Republicans’ corruption (2.63%), insulting or threatening Trump and Republicans (2.53%), criticizing Trump for too many firings (2.32%), criticizing Democrats’ handling of the Coronavirus pandemic, especially the government’s outreach with regard to lockdowns and restrictions (2.18%), fact-checking Trump and Republicans and pointing out their lies (2.16), condemning Democratic candidates’ treatment of women or children in a sexual way (2.14%), blaming Trump supporters for their blind support (1.92%), pointing out Trump’s foolish statements and temper-tantrums (1.70%), portraying Biden as the likely and deserving election winner (1.70%), accusing Trump and Republicans of racism, supremacy, sexism, homophobia, and transphobia (1.65%), offering theories about the origin or cause of the Coronavirus (1.63%), stating electability factors (1.63%), criticizing Trump and Republicans’ foreign policies (1.56%), criticizing Black Lives Matter protests (1.40%), condemning Republican candidates’ treatment of women or children in a sexual way (1.34%), reporting on election polls and results (1.22%), supporting Coronavirus restrictions (1.16%), criticizing China for its handling of the Coronavirus (1.15%), pointing out Biden and Democrats’ corruption (1.12%), and introducing drugs or treatments for the Coronavirus (1.08%). The remaining intents, each make less than 1% of Tweets.

One of the main challenges in front of OSNs is to understand the extent, topics, and trends of political discourse on their platforms and to minimize the amplifying effect they might have in spreading disinformation and fostering extremism. Considering the large number of users, massive number of posts, and variety of political topics, it is not possible to understand political discourse on OSNs as a whole. There needs to be a more detailed and organized description of topics. The data-driven and human-supervised classes of intent for political Tweets in this study provide this framework. The next challenge would be to detect the trending topics, disinformation, and extremism, and to measure their prevalence on OSNs, in close to real time. OSNs are increasingly resorting to artificial intelligence and machine learning for this purpose. Such automatic approaches are supervised in nature and require a well-defined set of classes and Tweets for training. The developed framework in this study could be used for designing and training machine learning models that would automatically classify Tweets based on their intent, political affiliation, disinformation, and extremism.

6. Conclusions and Future Directions

A data-driven framework for classifying the intent of political Tweets, along with analytics regarding their factuality, extremism, and political affiliation were developed based on a thorough study of 40,000 original Tweets, related to the U.S. 2020 presidential election. As a result, 171 classes of intent were defined. The results showed that disinformation and extreme opinions are more common among rightist Tweets than leftist Tweets. Leftist disinformation and extremism mostly insult or threaten Trump and Republicans, accuse them of mistreating women and children in a sexual way, and criticize their handling of the Coronavirus pandemic. Rightist disinformation and extremism mostly insult or threaten Biden and Democrats and accuse them of corruption, propagate false information about Democrats’ handling of the Coronavirus, propagate deep state and QAnon conspiracy theories, accuse left-wing media of bias, and criticize Black Lives Matter protests in an extreme way. Rightist extremism is mostly incited by QAnon conspiracy theories and false information about Democrats’ handling of the Coronavirus.

Overall, the Coronavirus pandemic is the topic of almost half of the Tweets, where 25.43% of Tweets express their unhappiness with how Republicans have handled this pandemic. This underscores Twitter users’ concern with the Coronavirus pandemic.

This study not only provides insight into OSNs’ role in politics and election and enlightens voters, politicians, and OSNs’ managers in this regard, but also establishes the foundation for different mechanisms that OSNs might take to provide more specific analytics regarding political discourse and flagging suspicious activities. Applying machine learning to detect the intent and party affiliation of political Tweets and whether they contain disinformation or extremism is based on the assumption that these classes are already defined. Our framework establishes a detailed foundation for these automatic approaches. Our future research direction is to invent mechanisms for real-time flagging and annotation of OSN posts, with respect to their political intent, affiliation, disinformation, and extremism.