The Impact of Autonomous Vehicle Accidents on Public Sentiment: A Decadal Analysis of Twitter Discourse Using roBERTa

Abstract

In the field of autonomous vehicle (AV) acceptance and opinion studies, questionnaires are widely used. Additionally, AV experiments and driving simulations are utilized. However, few AV studies have investigated social media, and fewer studies have analyzed the impact of AV crashes on public opinion, often relying on limited social media datasets. This study aims to address this gap by exploring a comprehensive dataset of six million tweets posted over a decade (2012–2021), and neural networks, sentiment analysis and knowledge graphs are applied. The results reveal that tweets predominantly convey negative sentiment (40.86%) rather than positive (32.52%) or neutral (26.62%) sentiment. A binary segmentation algorithm was used to distinguish an initial positive sentiment period (January 2012–May 2016) followed by a negative period (June 2016–December 2021), which was initiated by a fatal Tesla accident and reinforced by a pedestrian killed by an Uber AV. The sentiment polarity exhibited in the posted tweets was statistically significant (U = 24,914,037,786; p value < 0.001). The timeline analysis revealed that the negative sentiment period was initiated by fatal accidents involving a Tesla AV driver and a pedestrian hit by an Uber AV, which was amplified by the mainstream media.

1. Introduction

Social media has become a major communication tool for widely sharing opinions [1] across countries, transforming the Web into a dynamic information exchange arena [2]. For example, platforms such as Twitter (now known as “X” since 23 July 2023) have facilitated unity around important social issues. The #MeToo movement is a well-recognized example [3]. However, this widespread dissemination has raised concerns regarding the infodemic problem of false information sharing [4].

Social media has opened a potentially valuable Pandora’s box for the research community. Using deep learning and natural language processing, millions of tweets can be scrutinized and explained. For this particular task, sentiment analysis is widely employed in various research areas, such as research on vaccine hesitancy [5], monkeypox outbreaks [6], Asperger syndrome [7], and climate change [8].

In the field of autonomous vehicle (AV) acceptance and opinion studies, questionnaires are widely used methodologies, and include sociodemographic variables, which are used to define the level of AV acceptance across sex, income, and social class, for example [9,10,11,12]. Additionally, AV experiments [13,14] and driving simulations [15] have also been utilized. However, some rare AV studies have investigated social media platforms such as Twitter [1], TikTok [16], and YouTube [17]. Even fewer studies have analyzed the impact of AV crashes on public opinion [16,18,19], often relying on limited social media datasets. For example, Jefferson and McDonald [19] analyzed 11,164 tweets over a six-day period, whereas Jing et al. [16] scrutinized 16,635 comments extracted from TikTok and 9879 comments from Sina Weibo, a Chinese social media platform.

This study aims to address this gap by exploring a comprehensive dataset of six million tweets posted over a decade using sentiment and text content analysis tools. These datasets enable a thorough investigation of sentiment to comprehensively assess the impact of technological uncertainty, particularly when human lives are at risk.

2. Methods

The methodological stages of the study are summarized in Figure 1, and the corresponding explanations, from data collection to data analysis, are presented in the following subsections.

2.1. Data Collection

English messages posted on Twitter (tweets) discussing AVs between 1 January 2012 and 31 December 2021 were collected using API V2 Academic Research, utilizing pertinent keywords related to AVs (see Supplementary Material S1). A total of 6,773,504 tweets were subsequently extracted and saved in the MongoDB database.

2.2. Data Preprocessing

As the aim of this study is to analyze sentiment evolution on Twitter, the dataset was filtered using a specific emotion and valence dictionary (see Supplementary Material S2), and only tweets containing at least one emotional or valence word were retained. After removing duplicates, bot users and tweets with URLs, 3,379,636 messages were filtered to obtain a total of 509,069 tweets. A final cleaning was conducted to remove special characters (&gt, #, @, –, and |).

2.3. Data Labeling

Approximately 1200 tweets were randomly selected from the filtered final dataset and were assessed by two experienced individuals. Three labels were assigned to obtain a multiclass classification set (see Supplementary Material S3):

(0) “Negative” for tweets containing negative sentiments, such as distrust toward AVs or concerns about job losses;

(1) “Neutral” for tweets conveying neutral or informative content, such as commercial advertisements from car manufacturers, information about AV events, or informative statements without valency or an emotional perspective;

(2) “Positive” for tweets expressing favorable sentiments toward AVs, such as willingness to use them.

Notably, owing to the European General Data Protection Regulation (GDPR) [20], we are unable to provide examples of the tweets extracted from the dataset.

2.4. Artificial Neural Network Labeling

The pretrained “Twitter-roBERTa-base for Sentiment Analysis” model was selected and obtained from Huggingface [21] for training with multiclass labeling. This model was trained on approximately 58 million English tweets without URLs and fine-tuned specifically for sentiment analysis [22]. The manual labels were used for training, validation, and testing. The model achieved an accuracy of 0.69 and an F1-score of 0.69 for the test dataset. The 507,873 remaining unlabeled tweets were subsequently analyzed and automatically labeled using the trained RoBERTa model.

2.5. Data Analysis

Two types of analyses were conducted: (1) sentiment analysis with the automated labels obtained with the neural network model and (2) text mining analysis focusing on the content. To prepare the text, a dictionary of relevant words (see Supplementary Material S4), including misspellings, was compiled to filter the 13.53 million words (including 692,950 unique words). After rectifying misspellings and performing filtering, 3.04 million words (with 7699 unique words) were retained for text analysis and the construction of knowledge graphs. Note that mentions and hashtags were kept in the text as informative data for the purpose of the study. The knowledge graphs were created using Gephi 0.10 software. The two types of analyses were conducted using Jupyter Notebook and several Python libraries (Pandas, NumPy, Pyreadstat, Seaborn, Matplotlib, NLTK, re, Scikit-learn, Wordcloud, SpaCy, Stanza, SciPy, Ruptures, and NetworkX) for the text mining tasks (stemming, lemmatization, counting, vectorization, and text visualization), statistical analysis, text filtering, and graph construction.

3. Results

3.1. Tweet Count

The 507,873 analyzed tweets were posted by 299,468 unique users (i.e., tweeters) and contributed to 432,467 unique conversations. The most influential tweet received 45,927 likes and 8286 retweets. Over the studied decade (2012–2021), an average of 4232 tweets were posted per month, with a monthly range of 299 to 17,090 (Figure 2).

3.2. Sentiment Analysis

Most of the tweets from the studied decade were negative (40.86%) (

Table 1. Sentiment of analyzed tweets.

Sentiment	Number of Tweets	%
Negative	207,508	40.86%
Positive	165,157	32.52%
Neutral	135,208	26.62%

). Negative tweets are associated with ‘problem’, ‘kill’, ‘death’, ‘crash’, ‘technology’, ‘Tesla’ or ‘Google’. Conversely, positive tweets are associated with ‘hope’, ‘wait’, ‘future’, ‘safe’, ‘love’, ‘great’ and ‘Tesla’.

A sentiment polarity analysis was conducted to evaluate the fluctuations in public sentiment toward AVs over time (Figure 3). The analysis employed a scale ranging from −1 for negative sentiment to +1 for positive sentiment. To identify significant shifts in public opinion, a binary segmentation algorithm was applied to the time-series data using the Ruptures Python library. The binary segmentation algorithm [23] is a multiple change point search method [24] that serves as an approximation with “an O (n log n) computational, where n is the number of data points” [25]. This algorithm operates by recursively partitioning the data into segments and identifying points where significant changes occur in the statistical properties of the data [24] (see Supplemental Material S5 for further methodological details). This methodological approach facilitated the detection of critical junctures in public sentiment regarding AV technology.

Two periods of sentiment are evident in the data: an initial positive period from 1 January 2012 to 22 June 2016, and a subsequent negative period from 23 June 2016 to 31 December 2021. Each period comprises two subperiods, labeled A and B, characterized by a decreasing average sentiment score (Figure 4).

During the first period, the monthly average sentiment polarity was positive (sentiment score = 0.046; N = 114,140 tweets). However, sentiment shifted toward a more negative trajectory during the second period, with the monthly average sentiment polarity decreasing to −0.121 (N = 393,733 tweets) (see Supplementary Material S6).

A Mann-Whitney U test indicated a statistically significant difference in sentiment polarity between the two periods (U = 24,914,037,786; p value < 0.001), and a Kruskal-Wallis H test indicated the same results for the four sentiment periods (H = 4755.07; p value < 0.001) (Figure 4).

3.3. Text Mining

To better understand the content exchanged and discussions among tweeters, a text mining analysis of the relevant selected words was conducted. The term frequency-inverse document frequency (TF-IDF) score of the most important terms in the filtered dataset indicated that different terms were predominant in the two periods (Figure 5). ‘Google’ was more likely to be mentioned in the first period than in the second period. Conversely, ‘Tesla’ was most cited in the second period (see Supplementary Material S7 and S8). The term ‘kill’ was the first negative word in the top 20 most popular terms in the second period.

Some words were common in both periods, such as ‘technology’, ‘traffic’, ‘life’, ‘accident’, ‘Uber’, and ’Google’. Others were specific to the first period, such as ‘love’, ‘hope’, ‘today’, and ‘interesting’. In contrast, the second period had new words such as ‘Tesla’, ‘Tesla autopilot’, ‘artificial intelligence’, ‘kill’, and ‘robot’. This TF-IDF analysis identifies the main words used during conversation about AVs and provides insight into how negative terms influence period 2. Tesla, prominent in the second period, records a polarity score of −0.18, whereas Google, also prominent in the first period, achieves a positive score of 0.02 (see Supplementary Material S6). Common terms across both periods, such as ‘technology’, are mentioned in positive tweets during the first period (0.07) and in negative tweets during the second period (−0.09).

3.4. Knowledge Graphs

To deeply explore the connections between these words, knowledge graphs [26] were constructed. The knowledge graph facilitates this objective by employing nodes, representing words, and edges, which denote the relationships between them, to effectively capture semantic associations. Tweets were tokenized into individual words, and edges were established between the words within each tweet. Figure 6 shows that the content varied across the two sentiment periods.

During the first positive period, the conversations revolved around the Google company, accompanied by optimism stemming from advancements in smart AV technology (turquoise class). AVs were anticipated to enhance road safety, facilitate smooth traffic flow (green class), and reduce accidents caused by human drivers (red class), ultimately saving lives (pink class).

In contrast, the second negative period was characterized by discussions focused on companies such as Tesla and Uber, as well as artificial intelligence embedded in AV technology (turquoise class). However, this optimism was overshadowed by concerns surrounding pedestrian and driver fatalities involving Tesla and Uber AVs (red class). Additionally, apprehensions regarding job losses resulting from the automation of human tasks were highlighted, as indicated in the smaller orange segment of the graph.

3.5. Turning Point Events

The discussion about AVs on Twitter initially began with a high level of positive sentiment reflecting hope regarding preserving lives on the road and smart technology development led by Google (Figure 7).

The first notable increase in tweet activity coincided with the approval of AV laws in Nevada and California in 2012, which occurred in February and September, respectively. A second wave of tweeting occurred in May 2014 when Google announced its plan to launch 200 AVs on the roads. Following this announcement, the average sentiment polarity decreased by approximately 80.65% (from 0.124 to 0.024). However, the most significant impact occurred after the first fatal accident involving the driver of a Tesla car equipped with an autopilot function in the United States on 7 May 2016. The decline in emotional polarity then reached 358.33% (from 0.024 to −0.062) during this first part of the negative period, which spans from 2016 to 2018. Finally, an accident involving a pedestrian killed by an Uber AV on 18 March 2018 generated a large surge of tweets, marking the onset of the most negative sentiment period of the decade regarding Twitter users’ opinions about AVs. The sentiment polarity reached −0.147, representing a decrease of 137.10% (from −0.062 to −0.147).

4. Discussion

This study explores the sentiment content of tweets related to AVs over the past decade (2012–2021). The findings reveal that tweets predominantly convey negative sentiment (40.86%) rather than positive (32.52%) or neutral sentiment (26.62%), which is consistent with previous research [18,27]. Sentiment polarity analysis was used to differentiate two sentiment periods: an initial positive period from January 2012 to May 2016 and a subsequent negative period from June 2016 to December 2021. The sentiment polarity exhibited in the posted tweets is statistically significant (U = 24,914,037,786; p value < 0.001).

A timeline analysis of events revealed that the negative sentiment subperiods were initiated by fatal accidents involving a Tesla AV driver and a pedestrian hit by an Uber AV. The sentiment analysis revealed that these specific fatal accidents had an impact on the opinions of Twitter users. However, previous AV accidents did not have such an effect on opinion. For example, 11 minor accidents in November 2015, the first Tesla accident in China in January 2016, and a Google car hitting a bus in February 2016, received less attention. When fatal American accidents were widely covered in the mainstream media, including The Guardian [28,29] or The New York Times [30,31], a wave of negative sentiment was evident in the Twitter dataset.

This study demonstrates that public opinion changes over time. Initially, when AV technology was perceived as a promising life-saving innovation bridging the gap between fiction and reality, sentiment polarity was positive. Conversely, when AVs began sharing the road with humans and were involved in rare accidents, the perceived risk erased hope, concerns about safety arose, and sentiment polarity shifted to negative. While Wicki’s study [32] investigating the effects of AV accidents on families indicates that the negative impact is transient, our study of social media suggests that the effect appears to be persistent and progressively negative over time.

The main result of this sentiment analysis is that negative publicity spread across mainstream media and social platforms can generate fear and reluctance toward technology, as history has shown [33,34]. As technology is increasingly released to the market before full development, some innovations may be slowed or halted due to malfunctions, particularly if they could cause fatalities.

5. Limitations

While our analysis considers various factors influencing sentiment, the dynamic nature of social media discourse may introduce biases that could impact the interpretation of sentiment trends. Furthermore, the variability in user engagement and the shifting landscape of public discourse surrounding autonomous vehicles may constrain the generalizability of our findings beyond the specific time frames examined.

Given that autonomous vehicle (AV) tweets contain a substantial amount of informative and promotional content, the textual data were filtered using emotion and valency dictionaries. Consequently, the analyzed content is not representative of all discussions, but rather focuses solely on the emotional aspects of the discourse. This limitation constrains the scope and generalizability of the conclusions drawn. Tweet sentiment analysis presents significant challenges due to the brevity of textual data (with a mean of 160 characters per tweet). This constraint may account for the F1 score of 0.69 achieved with the RoBERTa pretrained model. Despite these limitations, the model’s performance is comparable to or surpasses that of other studies in tweet sentiment analysis employing manual labeling and the RoBERTa model. For example, Trivyza [27] reported an F1 score of 0.46, whereas Benítez-Andrades et al. [35] reported an F1 score of 0.74. These comparable results across various studies underscore the persistent challenges in Twitter sentiment analysis, suggesting that the current model’s performance falls within an acceptable range for this specific domain. Nevertheless, the reliability of the conclusions warrants scrutiny. The analysis of a filtered large volume of data (509,069 tweets), coupled with binary segmentation and rigorous statistical analysis, facilitates the identification of reliable patterns and mitigates the impact of potential misclassifications. This approach suggests that false labeling may be considered mere noise in the dataset rather than a significant confounding factor.

6. Conclusions

The present study represents a significant advancement in understanding public acceptance and rejection of autonomous vehicle (AV) technology. Through the analysis of an extensive dataset comprising tweets posted over a decade (2012–2021), the results reveal a predominance of negative sentiments over positive or neutral expressions. However, the timeline analysis, conducted using binary segmentation techniques, indicates that public opinion initially leaned positive, highlighting the anticipated benefits of AV innovation. From 2016 onward, sentiment shifted toward negativity following the Tesla AV accidents reported by mainstream media. The perception of AV technology then transitioned from highly positive to predominantly negative as public concerns increasingly aligned with the technical challenges reported by the media. This trend also illustrates how traditional media can amplify legitimate public concerns and shape online sentiment toward AVs.

To further elucidate the impact of technological failures on public acceptance, additional research will explore the relationship between AV performance and individuals’ perceptions of reliability using qualitative interview data.