Backfire Effect Reveals Early Controversy in Online Media

Abstract

The rapid development of online media has significantly facilitated the public’s information consumption, knowledge acquisition, and opinion exchange. However, it has also led to more violent conflicts in online discussions. Therefore, controversy detection becomes important for computational and social sciences. Previous research on detection methods has primarily focused on larger datasets and more complex computational models but has rarely examined the underlying mechanisms of conflict, particularly the psychological motivations behind them. In this paper, we propose a lightweight and language-independent method for controversy detection by introducing two novel psychological features: ascending gradient (AG) and tier ascending gradient (TAG). These features capture psychological signals in user interactions—specifically, the patterns where controversial comments generate disproportionate replies or replies outperform parent comments in likes. We develop these features based on the theory of the backfire effect in ideological conflict and demonstrate their consistent effectiveness across models and platforms. Compared with structural, interaction, and text-based features, AG and TAG show higher importance scores and better generalizability. Extensive experiments on Chinese and English platforms (Reddit, Toutiao, and Sina) confirm the robustness of our features across languages and algorithms. Moreover, the features exhibit strong performance even when applied to early-stage data or limited “one-page” scenarios, supporting their utility for early controversy detection. Our work highlights a new psychological perspective on conflict behavior in online discussions and bridges behavioral patterns and computational modeling.

1. Introduction

The definition of controversy or conflict in communication is the interaction of interdependent people who perceive the opposition of goals, aims, and/or values and who see the other party as potentially interfering with the realization of these goals [1]. Debates about religion, politics, intellectual ideology, and even personal preferences occur online and offline daily, and they can disrupt effective communication, leading to cognitive polarization, extremist behavior, and even war [2]. Therefore, ideological conflict has been one of the stresses in modern society that permeates all human activities.

The proliferation of online media has created the potential for more conflicts, and this can be attributed to the nature of the Internet. First, the Internet breaks through traditional geographical restrictions, allowing users with different cultural traditions, religious beliefs, and living habits to contact each other directly, resulting in unprecedented conflicts [3]. Second, the Internet makes users feel anonymous, resulting in the lack of necessary personal restraint in the online discussion process. Third, online media, especially social media based on user-generated content, are leaner and lack the necessary regulation, which has led to the proliferation of disinformation and further intensified conflicts [4]. Finally, artificial intelligence technology in online media could also spark more conflict [5]. These characteristics may result in volatile conversations that are more dynamic in the number of participants and opinions. While this can allow users to engage with each other and discuss differing viewpoints respectfully, it can also lead to escalated disagreements among users.

Given the universality and importance of online conflict, its research has attracted attention from both the social and computational sciences. Research from the social sciences has usually aimed at special controversial issues around critical politics and ideological topics, such as liberalism, equity, and environmental protection [6]. Social science mainly pays attention to the social, psychological, and cultural extension and connotation of these controversial issues and how users participate in discussing these issues [7]. In contrast, computational science has primarily focused on creating detection models in online media [8]. Facing the vast content of online media, efficient and accurate controversy detection methods are the technical basis for carrying out related research and the premise for effective conflict management.

Current controversy detection models are trained using features such as natural language, social networks, and user interactions to identify conflicting content at different levels, including topic-level, post level, and interaction-level [9]. However, the existing research still has certain limitations. With the development of artificial intelligence technology, natural language processing (NLP) methods have become the most natural choice for controversy detection. Due to natural languages’ inherent complexity and uncertainty, these models depend highly on language types and the pertinence of corpus [10]. As a result, these models cannot be used directly in other languages or topics. Unlike language-based models, detection models based on social networks and user interaction have better generalizability [11,12,13]. However, they usually exhibit severe data dependence, such as large-scale friend relationships or complete discussion processes. In multilingual and cross-platform environments, the challenge becomes more complex as models must cope with both behavioral variance and linguistic diversity. Prior studies, such as the use of artificial intelligence techniques for multilingual web traffic forecasting [14], have shown the potential of AI to generalize across languages. Inspired by this, our work also aims to design features that are robust across linguistic contexts, avoiding dependency on language-specific processing while preserving detection effectiveness. Meanwhile, the dilemma between universality, usability, and effectiveness requires further development of existing detection models, especially to discover more effective representations of online conflicts. The backfire effect, a cognitive phenomenon in which exposure to opposing views strengthens rather than weakens one’s original beliefs, has been extensively studied in recent years. It has been shown to amplify polarization in social media through mechanisms such as biased assimilation and homophily, as formalized in recent opinion dynamics models [15,16]. We hypothesize that the backfire effect may also manifest in measurable patterns of user behavior in real-world media platforms.

We aim to resolve this dilemma at the post level (whether a piece of news or a post will create conflict) by finding new conflicting features (Figure 1). Instead of analyzing and detecting a post using all the information of the post, we use the data that can be obtained within the “one page” containing the content and related comments of the post to avoid relying on massive data. At the same time, we use the post and the comment data with the earlier posting time to achieve early controversy detection. More thoughtfully, we consider the diversity of online media (including forums and news websites) and only select commonly available data for the various platforms. By considering the psychological mechanisms behind features, we have introduced two novel features for controversy detection across media platforms from different languages and types. We quantify the value of the new features by comparing the discernibility with other frequently used features. The new features can facilitate the development of better detection models, and this feature-mining method based on psychology and behavior also provides new pathways for future social media research. The main contributions of this paper include the following:

(1)

Propose two novel psychological features—ascending gradient (AG) and tier ascending gradient (TAG)—based on the backfire effect to capture ideological resistance in user interactions for effective controversy detection.

(2)

Demonstrate the robustness and generalizability of AG and TAG across multiple platforms (Reddit, Toutiao, Sina), languages (English and Chinese), and classification algorithms, outperforming traditional features.

(3)

Validate the practical utility of the proposed features in early-stage and limited-data scenarios (e.g., “one-page” content), enabling efficient and interpretable controversy detection for real-world applications.

The remainder is organized as follows. Section 2 provides an overview of the related work on controversy detection, focusing on different categories of features. Section 3 outlines multilingual and multi-platform datasets, candidate features, and the detection method. Section 4 presents the analysis of the experiments. Finally, Section 5 summarizes the key findings and highlights future research directions.

2. Related Work

Many features have been used for controversy detection, which can be divided into four categories: user characteristics, text-based features, social networks, and discussion interactions. In this section, we briefly summarize the research on these features.

2.1. User Characteristics

Reviewing previous research, distinct population characteristics can lead to different types of participation in conflict. For example, Emily and Davinia showed that there was a big difference between males and females when it came to aggressive behavior [17], and Holt and DeVore found that men were more likely to report using force than women in individualistic cultures and with regard to organizational role, men were more likely than women to choose a forcing style with their superiors [18]. Triandis demonstrated that the differences between culture and race had profound implications for the probability of conflict and the type of conflict [19]. Cheng et al. [20] showed that antisocial behavior in online groups is a relatively stable individual difference. Levy et al. [21] also proved that user activity is also closely related to conflict behaviors. In addition, the previous behavior of the users participating in the discussion can also be used to detect a future conflict. Saveski et al. [12] found that the structural characteristics of the conversation were also predictive of whether the next reply posted by a specific user would be toxic or not. These works emphasize the predictive capacity of user characteristics. However, they suffer from practical limitations: such characteristics often require access to external data beyond the discussion page, may violate user privacy, and are not always available in anonymous environments.

2.2. Text-Based Features

In controversy detection, the text data of posts or comments are usually easy to obtain, which is conducive to fully mining sentiments, emotions, and opinions [9,22,23,24]. These papers have analyzed the intensity of emotions and sentiments, showing that these text-based features are strong indicators of controversy. Weingart et al. [25] analyzed conflict spirals in the workplace that were influenced by conflict expressions varying in directness and oppositional intensity. Zhang et al. [26] predicted from the very start of a conversation whether it would get out of hand within Wikipedia talk page discussions through various text-based features such as politeness strategies and prompt types. Text-based methods rely fundamentally on NLP techniques, which are inherently limited by language specificity and corpus dependency. Moreover, compounded by social media’s informal expressions and short texts, it reduce sentiment analysis accuracy across languages and platforms—motivating language-independent alternatives.

2.3. Social Networks

In addition to user characteristics, researchers study the conflict commenter’s social network structure. Conover et al. [27] demonstrate the existence of controversial structures in Twitter through network modularity and graph partitioning based on user relationships and behavioral relationships in social networks. Morales et al. [28] quantified the controversial nature of topics through opinion leaders in the network structure. Akoglu et al. [29] constructed a bipartite graph based on user relationships to quantify the contentiousness of information. Coletto et al. [11] studied users’ connections in the context of controversial threads. To do so, they analyzed local network patterns of user-follower and user-reply graphs. Their findings showed that controversial interactions are less likely between users who follow each other on social media. de Dreu argued that conflicts occurred more often in intergroups than in interpersonal [30]. Many of these conflicts are identity-based, in which people believe that the group or subgroup they identify with (e.g., ethnicity, race, religion, or political party) is superior to an outgroup [31]. While many conflicts originate from differences between intergroups, these intergroup conflicts can evolve into interpersonal ones [32], and frequent pleasant interactions can reduce intergroup conflicts [33].

Furthermore, some scholars combine network structure with text semantics. Zhong et al. [34] proposed a controversial microblog detection algorithm based on a Graph Convolutional Network. This algorithm builds a topic-microblog-comment network based on behavioral relationships and combines the textual information of topics and comments to detect controversial details. Benslimane et al. [35] proposed a method for contentious information detection based on graph structure and textual features. The technique embeds graph representations (including text features) into feature vectors [36], considers the attention mechanism when calculating user nodes, fuses the importance of neighbor nodes, and finally performs the classification task through the Graph Neural Network. These methods uncover underlying group dynamics but often rely on large-scale and structured social network data, which is unavailable on many platforms, such as news websites. Our study seeks to avoid such dependency by proposing lightweight features extracted from visible interactions.

2.4. Discussion Interactions

In addition to social relations, users will also discuss and interact through comments below the post. The structure of this interaction has also been shown to be related to the conflict. Coletto et al. [11] not only analyzed the relationship between user-follower and user-reply networks but also investigated intensity through a different perspective by assuming controversial topics may generate “dense” discussions in time so that the inter-reply rate for these conversations is lower (i.e., more rapid replies) than those of non-controversial ones. Such interaction structures capture valuable behavioral signals that reflect emotional involvement, disagreement, and user polarization. Discussion interaction contains the psychological and behavioral characteristics of users expressing opinions on conflicting issues, and its potential needs further exploration. However, existing methods based on interaction features often rely on manually defined structural heuristics, which may not fully capture the latent psychological patterns associated with ideological conflict. Additionally, few works explicitly connect these behaviors to cognitive theories, such as the backfire effect, which explains how users react to opposing views. Motivated by this gap, our study aims to uncover and formalize behavioral patterns within discussion interactions that are theoretically grounded and computationally lightweight.

3. Methodology

Motivated by the limitations of existing approaches and the need to better capture psychologically behavioral signals in online interactions, we construct a lightweight, effective controversy detection model, as illustrated in Figure 2. The proposed model is designed to uncover latent indicators of ideological disagreement embedded in discussion interactions. The framework consists of three major components: Data Processing, Feature Extraction, and Model Building. This section will elaborate on each component of the proposed model in detail. The effectiveness of the model will be further validated through specific tasks.

3.1. Data Processing

To evaluate the discriminative power of features in different languages and types of online media, we collected three datasets from various sources: Reddit (English), Toutiao (Chinese), and Sina (Chinese). Each dataset contains posts from multiple sets of conflicting and non-conflicting topics from their respective platforms. Our methodology is not language-dependent, as it focuses on user behaviors such as likes and replies. As long as a platform supports these interactions, the approach is applicable. In this study, we use Chinese and English media as representative examples.

The English-based media platform is Reddit. For the Reddit platform, we use the May 2015 public dataset from the kaggle.com platform. We filtered controversial topics from two subreddits using keywords according to the list from the library of Shippensburg University. The two topics mentioned above, Gun and War, and the content under the “DebateReligion” subreddit make up the three controversial topics. Meanwhile, we get three mild topics of Shopping, Scenery, and Music around the three subreddits “Random_Acts_Of_Amazon”, “Watches”, and “Guitar”. The dataset contains the complete information of posts, comment information, and the relationship between each comment. To ensure that the posts under each topic have a complete relationship structure and diverse information attributes, we mainly focus on posts with more than 50 comments. Finally, the dataset contains 542 news items and 110,176 comments (The main data in

Table 1. All datasets. Three Chinese and English media platforms (Reddit, Toutiao, Sina) are included, and six topics are extracted from each platform. Each topic contains the attributes of controversy, the total number of posts, and comments. Reddit is the main experimental data. Toutiao and Sina are used for experimental testing.

Type	Media	Topic	Controversy	Post	Comment/Like
Main Data	Reddit (English)	Gun	✓	79	19,371/150,772
		War	✓	66	17,556/139,894
		Religion	✓	122	25,663/57,355
		Shopping	✗	111	26,503/40,831
		Scenery	✗	71	7,805/38,476
		Music	✗	93	13,278/83,951
Type	Controversy	Toutiao (Chinese)		Sina (Chinese)
		Topic	Post/Comment/Like	Topic	Post/Comment/Like
Test Data	✓	Huawei	111/25,317/63,838	Huawei	118/23,578/635,701
		NBA	104/22,695/84,768	NBA	145/20,608/378,094
		Football	96/21,747/44,494	Football	171/23,617/542,666
	✗	Life	104/13,434/96,734	Science	84/9,561/206,664
		Food	115/26,518/21,068	Music	78/10,943/94,213
		Travel	117/10,014/33,769	Movie	132/16,155/197,196

).

The Chinese-based media platforms are Toutiao and Sina. Toutiao, or Jinri Toutiao, is a news and information platform based on user-generated content and recommendation algorithms. The dataset includes news information, comment information, and the relationship of comments. The period of the data is from January 2019 to December 2019. In Toutiao, we choose the topics of Huawei, NBA, and Football that have apparent controversy and the topics of Life, Food, and Travel with minor controversy. The raw data of this dataset is collected by the web crawler, and then we clean the raw data to remove the empty and duplicate values. In the end, there were 647 news articles and 119,725 comments in the dataset. Sina is one of the most influential news platforms on the Chinese Internet. The period of crawled data is from January 2019 to December 2019. We select the same controversial topics as the Toutiao, as well as mildly controversial topics such as Science, Music, and Movies, for a total of 728 news items and 104,462 comments (the test data in Table 1).

Our dataset has a clear hierarchical structure. Topics (subreddits) allow users to come together to discuss topics of interest, share relevant content and resources, interact with others, and build a community focused on a specific topic. Posts are the body for our subsequent analysis, containing information such as the post topic, the post time, and the user ID. Comments are the key unit of our analysis and contain information such as comment text, comment time, and number of likes.

This paper uses P to denote a post, that is

$$P=\left\{\mathrm{Topic},P_{id},P_t,T\right\}$$

(1)

where $P_{id}$ is the individual ID of a post, $P_t$ is the time when a post was published, and T is the post content.

We define a comment as C, that is:

$$C=\left\{P_{id},C_{id},C_t,C_l,T,C_{pid}\right\}$$

(2)

where $P_{id}$ is the ID of a post to which the comment belongs, $C_{id}$ is the comment ID, $C_t$ is the time that the comment was posted, $C_l$ is the number of likes for a comment, T is the textual content, and $C_{pid}$ is the ID of the parent text of the comment (i.e., the previous level of text to which the current comment belongs).

The form of a graph can be good for representing and understanding posts. We define a node set $V=v_0,v_1,v_2,\cdots ,v_n$, where $v_0$ represents the post P, $v_i(i\ne 0)$ denotes the comments, and n is the number of comments on the post. Then, we define the edge set $E=e_1,e_2,\cdots ,e_n$. Where, if the parent text ID $C_{pid}$ of node $v_j$ points to node $v_i$, then the connected edge $e_j=<v_i,v_j>$ is formed. From this, we can construct a comment graph $G=(V,E)$ for different data in terms of posts, and Figure 2a is a simple schematic diagram.

3.2. Feature Extraction

Based on multilingual and multi-platform datasets, we choose the common features among them as comparison objects. This web page usually contains the post title, content, and comments on the post and their replies. Therefore, we consider structural, interactive, and textual features. Structural features describe the tree-like organization of comments and replies, quantifying the discussion’s shape and connections. Interactive features capture user engagement and temporal dynamics within the discussion, such as reply times and comment frequency. Textual features analyze the emotions in both the main post and its comments. Note that we did not consider user characteristics, as this was far beyond the data presented by the information presentation page. We aim to identify conflicting posts with limited information quickly.

First, we consider features related to comment structure. A post and its comments can form a tree structure, where the root is the post and other nodes are comments, and these nodes are connected by comments or reply relationships. Based on the tree structure, we can extract the following features [37,38]:

• Size (n). The size of a tree corresponds to the number of nodes (without root) in that tree. For example, the size of the demo post in Figure 2 is 5.
• Depth (d). The depth of a node is the number of links from the node to the root. The depth of a tree is the maximum depth of the node in all nodes. In other words, the depth of a tree, d, with n nodes is defined as

  $$d=max\left(d_i\right),0\le i\le n$$

  (3)

  where $d_i$ denotes the depth of node i. For example, in Figure 2, $d=3$.
• Breadth (b). The breadth of a cascade is a function of its depth. At each depth, the breadth is the number of nodes at that depth. The breadth of a tree is its maximum breadth at all depths. For a tree with depth d, the breadth, b, is defined as

  $$b=max\left(b_i\right),0\le i\le d$$

  (4)

  where $b_i$ denotes the breadth at depth i. For example, in Figure 2, $b=2$.
• Average degree (k). In a tree, the degree of a node is only the number of direct children of a node. Therefore, the average degree of a tree is defined as

  $$k={\displaystyle \frac{{\sum }_{i=1}^n{k}_i}{n}},0\le i\le n$$

  (5)

  where $k_i$ denotes the degree of node i. For example, in the Figure 2, $k=1.2$.
• Structural Virality (v). The structural virality of a cascade, is the average distance between all pairs of nodes in a cascade. For a cascade with $n>1$ nodes, the virality v is defined as

  $$v={\displaystyle \frac{1}{n(n-1)}}\sum _{i=1}^n\sum _{j=1}^n{d}_{ij}$$

  (6)

  where $d_{ij}$ denotes the length of the shortest path between nodes i and j. For example, in Figure 2, $v=1.1$.

Second, we considered text-based features, including the following:

• Intensity of comment emotion (${\mathit{s}}_{\mathit{c}}$). We combine two sentiment methods, Vader and SnowNLP, to achieve efficient semantic analysis of Chinese and English [39,40]. Based on the sentiment dictionary, a sentiment score $c_i$ can be calculated for each comment (node i), and then the comment emotional intensity of a post can be obtained by the following equation:

  $$s_c={\displaystyle \frac{{\sum }_{i=1}^n{c}_i}{n}},0\le i\le n$$

  (7)
• Intensity of post emotion (${\mathit{s}}_{\mathit{p}}$). The emotional intensity of a post is also based on the above sentiment dictionary, and the sentiment score of an entire post’s content is calculated as $s_p$.

Third, the features of discussion interaction are also considered [11]:

• Reply time ($\mathit{t}$). For each reply link $e_x=〈i,j〉$ in a post, the time elapsed between the parent comment (node i) and its child (node j) is reply time ($t_{i}j$). We consider average ($t_{avg}$) and minimum ($t_{min}$) values of all $e_x$.
• Comment density ($\mathit{c}$). For a post, the time interval from its publication to the latest comment is $\delta$; then, the density of comments can be defined as

  $$c={\displaystyle \frac{n}{\delta }}$$

  (8)

  where n denotes the number of comments.

Unlike the previous features, we consider the number of likes and comments [21]. We first define $L_i$ to denote the number of likes received by comment i and consider the average number of likes received by all comments:

$$q={\displaystyle \frac{{\sum }_{i=1}^n{L}_i}{n}}$$

(9)

Second, we hypothesize that when users view a comment with an opposing opinion, they are more willing to press “like” to reply to criticize the comment. So, we define each reply edge as $e_x=〈i,j〉$, where i is the parent comment and j is the reply. The ascending gradient function is then defined based on whether the reply comment receives more likes than its parent:

$$g_a\left(e_x\right)=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{L}_i-L_j<0;\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(10)

For all edges, we can obtain the proportion of ascending gradient as

$$p_a={\displaystyle \frac{{\sum }_{x=1}^m{g}_a\left(e_x\right)}{m}}$$

(11)

where m is the number of reply edges in a post.

Furthermore, we expand the concept of ascending gradient to encompass the difference in the number of replies across comments, which we refer to as a “tier ascending gradient”. First define $R_i$ for each comment, i.e., the total number of comments that directly reply to comment i. Then, similar to Equations (10) and (11), we can obtain the functional representation of the tier-ascending gradient as

$$g_t\left(e_x\right)=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{R}_i-R_j<0;\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(12)

$$p_t={\displaystyle \frac{{\sum }_{x=1}^m{g}_t\left(e_x\right)}{m}}$$

(13)

All features are also summarized and described in Figure 2b. For convenience, we subsequently abbreviate the ascending gradient as AG and the tier ascending gradient as TAG and refer to them uniformly as psychological features (this will be analyzed in the experimental section to explain the psychological property of our proposed features). Finally, all the features and their descriptions that we have compiled are shown in

Table A1. Feature description.

No.	Name	Category	Symbol	Detailed Description
1	Size	Structure	n	Number of nodes in the comment tree
2	Depth	Structure	d	Maximum depth of the comment tree
3	Breadth	Structure	b	Maximum breadth of comment tree
4	Average degree	Structure	k	The average degree of the comment tree
5	Structural virality	Structure	v	The average shortest path of the comment tree
6	Minimum reply time	Interaction	$t_{min}$	The shortest time of all reply pairs
7	Average reply time	Interaction	$t_{avg}$	Average time for all reply pairs
8	Comment density	Interaction	c	Number of comments per second on post
9	Average ups	Interaction	q	Average of the number of likes of all comments
10	Comment emotional intensity	Text	$s_c$	Average of the emotional intensity of all comments
11	Post emotional intensity	Text	$s_p$	The emotional intensity of post content
12	Ascending gradient	Psychology	$p_a$	Percentage of difference in likes for all reply pairs
13	Tier ascending gradient	Psychology	$p_t$	The difference in the number of reply to all comments

.

3.3. Model Construction

LightGBM is a gradient enhancement framework based on the decision tree algorithm proposed by Microsoft Research in 2017 [41], which is widely used in classification or regression tasks. LightGBM optimizes the defects of Gradient Boosting Decision Tree (GBDT) that exist in high time and space consumption, supports efficient parallel training, and has the advantages of faster training speed, lower memory consumption, and a wider amount of processed data. It can also be called the GBDT algorithm using Gradient-based One-Side Sampling (GOSS) [41] and Exclusive Feature Bundling (EFB) [41], so first is a brief introduction to the core strategies of GOSS and EFB.

GOSS: The GOSS algorithm starts from the point of view of reducing samples, excludes most of the samples with small gradients, and uses only the remaining samples to calculate the information gain, realizing the reduction in the data volume while guaranteeing the balance of accuracy. Therefore, GOSS only retains the data with larger gradients when sampling data, but if all the data with smaller gradients are directly discarded, it will definitely affect the overall distribution of the data. Therefore, GOSS firstly arranges all the values of the features to be split in descending order of absolute value size and selects $a\ast 100\%$ of the data with the largest absolute value. Then $b\ast 100\%$ of the remaining smaller gradient data are randomly selected. This $b\ast 100\%$ of the data is then multiplied by a constant $(1-a)/b$ so that the algorithm focuses more on the under-trained samples and does not change the distribution of the original dataset too much. Finally, this $(a+b)\ast 100\%$ of the data is used to calculate the information gain.

EFB: In order to solve the sparsity of high-dimensional data, mutually exclusive features are bundled to achieve feature dimensionality reduction without losing information. The process is mainly divided into four steps:

(1)

Take features as the node of a graph, for the features that are not mutually exclusive are connected (i.e., there exist samples that are not 0 at the same time), and the number of samples whose features are not 0 at the same time are used as the weights of the edges.

(2)

Sort the features in descending order based on the degree of the nodes, with a larger degree indicating that the features are in greater conflict with other features (the less it will be bundled with other features).

(3)

Set the maximum conflict threshold K, traverse the existing feature clusters, and if it is found that the number of conflicts for the feature to be added to the feature cluster will not exceed the maximum threshold K, then the feature is added to the cluster. Otherwise, create a new feature cluster and add the feature to the newly created cluster.

(4)

Separate the original feature from the merged feature by adding an offset constant.

Among them, the first three steps solve the problem of determining which features should be bundled together, and the last step solves the problem of how to bundle (merge) multiple features into one.

In addition to this, the histogram-based decision tree strategy allows the algorithm to run with a smaller memory footprint and less computational cost. The leaf-wise strategy with depth constraints ensures the high efficiency of the algorithm while preventing overfitting. So, LightGBM is our main algorithmic model. To further clarify the implementation, we provide the pseudo-code of the entire method, as shown in Algorithm 1.

Algorithm 1 Controversy Detection Model.
Require: $\mathrm{dataset}=\{{\mathrm{post}}_1,{\mathrm{post}}_2,\dots ,{\mathrm{post}}_m\}$	▹ Collection of posts with comment trees and labels
Ensure: trained_model	▹ LightGBM classifier for controversy detection
1: // Initialize LightGBM with optimization techniques
2: model ← LightGBM
3: SetParam(model, “objective”, “binary”)	▹ Binary classification
4: SetParam(model, “boosting”, “gbdt”)	▹ Gradient Boosting Decision Trees
5: SetParam(model, “sampling”, “GOSS”)	▹ Gradient-based One-Side Sampling
6: SetParam(model, “bundling”, “EFB”)	▹ Exclusive Feature Bundling
7: // Process each post in the dataset
8: for each post ∈ dataset do
9:      features ← []	▹ Initialize feature vector
10:     // Extract structural features
11:     features[0] ← CommentCount(post)	▹ Number of comments (n)
12:     features[1] ← MaxDepth(post.tree)	▹ Maximum depth (d)
13:     features[2] ← MaxBreadth(post.tree)	▹ Maximum breadth (b)
14:     features[3] ← AverageDegree(post.tree)	▹ Average node degree (k)
15:     features[4] ← StructuralVirality(post.tree)	▹ Avg. path length (v)
16:     // Extract textual features
17:     features[5] ← AvgCommentSentiment(post)	▹ Avg. comment sentiment ($s_c$)
18:     features[6] ← PostSentiment(post)	▹ Post sentiment intensity ($s_p$)
19:     // Extract interaction features
20:     features[7] ← AverageReplyTime(post)	▹ Avg. reply time ($t_{avg}$)
21:     features[8] ← MinimumReplyTime(post)	▹ Min. reply time ($t_{min}$)
22:     features[9] ← CommentDensity(post)	▹ Comments per time unit (c)
23:     // Extract psychological features (novel contribution)
24:     features[10] ← LikeAscendingGradient(post)	▹ AG proportion ($p_a$)
25:     features[11] ← ReplyAscendingGradient(post)	▹ TAG proportion ($p_t$)
26:     AddTrainingSample(model, features, post.label)
27: end for
28: // Configure optimization parameters
29: SetGOSSParam(model, top_rate=0.2, other_rate=0.1)
30: SetEFBParam(model, max_conflict=5)
31: SetParam(model, “num_leaves”, 31)
32: SetParam(model, “learning_rate”, 0.05)
33: // Train model with GOSS and EFB optimizations
34: Train(model)
35: // Predict controversy probability
36: prob ← Predict(model, features)
37: return model, prob

4. Results and Discussion

In this section, detailed experiments are conducted around the task realization designed in Section 3, and the results are presented in three parts as follows: algorithm evaluation results, importance evaluation results, and application evaluation results.

4.1. Controversy Detection Results

If suitable features are chosen, even simple classification models can achieve excellent detection results. Based on this viewpoint, we first validate the enhancement effect of psychological features on different controversial topic detection algorithms, which include interval-based SVM, distance-based KNN, probability-based LR, tree-structure-based DT, integrated learning-based GBDT, and LightGBM. The detailed results, as shown in

Table 2. Algorithm evaluation results. The detection performance of different features in different classification models is evaluated on the Reddit dataset. The detection performance of different features in the LightGBM model is verified on the Toutiao and Sina datasets (The underlined numbers represent the optimal results with the different datasets).

Platform	Method	Feature
		Structure	Interaction	Text	Psychology
Reddit	SVM	80.98	80.37	80.98	93.25
	KNN	82.82	82.82	85.28	89.57
	LR	80.37	79.75	80.37	91.41
	DT	74.23	82.82	76.68	85.27
	GBDT	82.82	84.66	86.50	90.80
	LightGBM	84.05	84.05	85.28	93.25
Toutiao	LightGBM	77.84	89.18	88.66	90.21
Sina	LightGBM	78.22	82.18	82.67	83.66

, reveal the detection performance across four types of features—structure, interaction, text, and psychology—under each algorithm. The values in the table represent the classification accuracy (%) of each feature group under each algorithm. Across all classical classifiers, the best performance is achieved when using psychological features, with LightGBM and SVM both reaching 93.25% accuracy on the Reddit dataset. Compared with structure, interaction, and text features, psychological features consistently contribute the highest performance across almost all algorithms. When psychological features are removed, all algorithms show a clear performance drop, with a maximum reduction of over 10%, indicating their irreplaceable contribution. We further validate this trend on Chinese platforms using the current best-performing LightGBM algorithm. As shown in Table 2, psychological features again lead to the highest detection accuracy: 90.21% on Toutiao and 83.66% on Sina, outperforming all other feature groups. In the English platform (Reddit), text-based features also show strong support due to the maturity of NLP techniques in English, while in the Chinese datasets, text features perform less effectively, partly due to the limitations in sentiment analysis for short Chinese texts. This highlights the language sensitivity of text features, whereas our psychological features show cross-lingual robustness and generalizability. Unlike the volatility seen with structural, interactive, or text-based features—whose effectiveness can fluctuate with data scale or language—the psychological features maintain a stable and strong enhancement across algorithms and platforms. This stability underlines their potential as reliable indicators and suggests value for further exploration in future research.

4.2. Importance Evaluation Results

4.2.1. Quantitative Analysis

At the post level, we mixed all posts regardless of topic and assessed the importance of features using three classification algorithms. The results show that in all algorithms and platforms, the two features ($p_a$, $p_t$) based on the ascending gradient are far more critical than the others in distinguishing controversies and are generally in the top three positions of importance (Figure 3a–c). In the RF algorithm, the two features are in the first and second positions, respectively, with values up to 0.27 and 0.14. In the XGBoost algorithm, the gap between the first feature and the second feature can even reach nearly three times. The SHAP [42] values of each feature also show the same effect (Figure 3d), i.e., $p_a$ is generally in the first position, and psychology-based features are commonly in the top three positions. It is worth noting that in Figure 3a, some features (such as post-emotion, comment emotion, and structural breadth) show negative values. This is because the linear regression-based importance measure allows negative coefficients, indicating that those features are negatively correlated with the likelihood of a post being controversial. This also reflects the inherent volatility in the emotional and structural aspects of online content. Overall, for the discriminative ability of conflicting posts, the value of psychological features is significantly better than other features, while $p_a$ is generally better than $p_t$.

Second, we used the Kolmogorov–Smirnov test to compare the differences in the distribution of each feature across the three conflicting and non-conflicting topics of the Reddit dataset (Appendix A Figure A1). The results showed that among all five comment structure features, only breadth and structural virality differed significantly between conflicting and non-conflicting topics ($p<0.05$). Also, there were significant differences between specific topics in terms of size and depth. For text-based features, the sentiment intensity of comments mostly showed certain differences when comparing conflicting and non-conflicting topics, which is consistent with previous findings [23]. We found a significant difference in the distribution of average replay time and average ups for discussions between conflicting and non-conflicting topics, with slightly shorter phrases for controversial topics and non-controversial topics.

Among the different types of topics in the Reddit platform, for the gradient-related features (AG features in blue and TAG features in orange in Figure 4), there is a clear distributional difference, i.e., the feature values of non-controversial topics are generally lower than those of controversial features. Controversial topics such as “Gun”, “War”, and “Religion” have significantly higher AG and TAG values than non-controversial ones like “Shopping” or “Music”, indicating that user engagement—measured by likes and replies—increases more rapidly and persistently as discussions deepen. In particular, the average value of the AG feature in controversial topics is about 20% higher than that of non-controversial topics, which shows that the feature is a common situation among different types of topics. To verify that the difference in the average number of likes and replies to comments is also a common phenomenon across different platforms, we conducted a validation on two news platforms, Toutiao and Sina, whose results show a very obvious difference as well. Although these platforms mainly support flat comment structures, controversial topics such as “Huawei” and “NBA” still show noticeably higher AG and TAG values compared with neutral topics like “Food” or “Life”, while the latter often approach zero. Meanwhile, because the news platforms are mostly dominated by first-level comments, the feature magnitude of the controversial topics is a few times different from that of Reddit platforms, and the gradient feature values in terms of non-controversial topics are dominated by 0. In summary, the controversial topic gradient feature values are generally higher than the non-controversial topics, with significant fluctuations. These results demonstrate that gradient-based features effectively distinguish controversial content, even across platforms with differing comment structures.

To summarize, psychological features are significant and valuable in detecting controversies, and they can be applied to various media platforms in both Chinese and English.

4.2.2. Psychological Interpretation

Why does the ascending gradient perform well in discriminating between conflicting topics and posts? From a psychological point of view, the backfire effect could explain the emergence of the ascending gradient of likes and replies. A recent wave of studies suggested exposure to those with opposing political views may create backfire effects that stimulate the user’s cognitive resistance and exacerbate political polarization [15,43]. In theory, the deeper the level of comments, the more difficult it is to be exposed and the lower the likelihood of getting likes or replies. But in controversial posts, when some users (who may be silent by default) first see a comment that challenges their opinion at a lower level, and then, due to the backfire effect, they will be more insistent on their opinions. They will be motivated to seek psychological compensation. Therefore, when these users find that other users are refuting or fighting against this comment, they are more likely to be encouraged to express their desire to express it through likes or replies. In the upper level, comments that are consistent with the user’s point of view are not easy to get likes or replies from the type of users because they are not stimulated by the backfire effect. Since this excitation is consistent with user groups with opposing opinions, it leads to the emergence of ascending gradient situations in controversial posts. Due to this, we refer to the proposed gradient features as psychological features.

The backfire effect reminds us that in the discussion of controversial topics, users will be repeatedly exposed to opposite opinions, which will inevitably change their psychological state. It can be considered that the increase in emotional intensity and negativity in the discussion is related to this, but emotion recognition relies on natural language processing technology, which has certain limitations. In a word, the user’s mental state and behavior leading to conflict is a causal relationship, while identifying conflict based on techniques such as natural language processing is a process of data analysis. Next, we apply psychological features to several real-life application scenarios.

4.3. Application Evaluation Results

In view of the above sufficient validation, we focus on more efficient application scenarios applied to topic controversy detection for a small amount of hot information (one page) or partial information (early detection). We demonstrate that our approach achieves efficient controversy detection with minimal information while providing an interpretable analysis of its performance.

4.3.1. One Page

For each post page under the social platform, the first thing that is displayed is the highly liked comments to get an idea of the hot topics being discussed at the moment. As a result, most of the debates are also centered around the hot comments, while the hidden comments, due to low attention, have less influence on the judgment of topic conflicts. Therefore, we propose “one-page”, that is, using a small amount of intuitively visible hot information to achieve highly accurate controversy detection. For the experimental validation of hot comments, we focus only on comments with a high number of likes (top 20%) and their reply pairs in the current post. Meanwhile, the four structural features are not used because the global tree structure is lost. With only 20% of the hot comment information used, the LightGBM algorithm still has 87.12% detection accuracy with nine features. We again used the above three methods to assess the importance of features, and the results are shown in Figure 5a. Psychological features are generally in the top position, and their values are significantly better than other features.

As shown in Figure 5b, it is worth noting that the results suggest 0.2 may serve as a critical threshold where psychological features begin to exhibit strong differences. Although this effect is most prominent in the religion topic, it reveals a general pattern in feature dynamics. Specifically, TAG features dominate in the early stages, when only hot comments are available, due to their ability to capture surface-level engagement patterns such as reply density. In contrast, AG features become increasingly effective as more data accumulates, capturing deeper endorsement patterns such as the evolution of likes across reply layers. As the comment ratio increases, the inter-class variance of AG features expands while that of TAG features contracts. This demonstrates the complementary roles of AG and TAG: TAG is more informative with limited early data, while AG excels in full-data conditions, ensuring robust performance across different time stages. This dynamic also explains the difference in feature rankings between Figure 3 (full information) and Figure 5a (partial information).

We also explain the analysis in terms of the distribution of feature values (Figure 5c shows the case of posts with complete information on the left and the case of posts with hot information on the right), where the horizontal and vertical coordinates of the figure represent the two psychological features, and each point represents a post. It can be clearly seen that when the information is complete, the boundary between the two types of posts is clear, and the AG feature brings more distinctive differentiation. When the information is incomplete, a large number of non-controversial posts (in orange), are distributed all over the bottom, which makes the vertical TAG feature work more clearly than the horizontal AG feature. Overall, both AG features and TAG features can play an important role in controversy detection while also complementing each other in different situations.

4.3.2. Early Detection

The important findings described above also provide additional theoretical support for our realization of earlier conflict discovery. Because each comment of the data carries a posting time, we analyze the posts for different time periods. To do so, we sort all comments of each post by time, use the first 3 h of comments for the computation of features, and evaluate their controversy detection performance. In the time interval, each round linearly increases the number of comments by 3 h for multiple rounds of experiments, and in the features, each round adds different types of features on top of the existing ones, and the results are shown in

Table 3. Evaluate the controversy detection performance in different feature modes using comment information over different time periods (The bolded numbers indicate the size of the detection performance improvement, representing the best performance enhancement compared to previous results).

Mode (Feature Num)	Time
	3 h	6 h	9 h	24 h
Structure (5)	68.71	73.62	82.21	84.05
Interaction (9)	73.62	83.44	84.66	84.05
Text (11)	73.62	87.73	84.05	85.28
Phychology (13)	73.62 + 11.66 (85.28)	87.73 + 5.52 (93.25)	84.66 + 9.21 (93.87)	85.28 + 8.59 (93.87)

. It can be seen that around 3 h after the post is sent out, the accuracy of detecting whether it is a controversial article or not can reach 85.28% with the possession of psychological features, which is close to reaching the late state of other feature modes. Using only 6 h of data approximated the performance of using 24 h of data to achieve 93.25% controversy detection performance. In addition to this, the detection performance of the psychological mode has a clear advantage in almost all stages, and the improvement becomes more obvious the further back we go (after “+” in the table are the values of the current column’s improvement relative to the best accuracy). Our feature model generally achieves its best detection performance in the first and middle phases and remains stable until the event is unattended, reflecting the model’s ability to recognize conflicts earlier as well as the stability of its features.

5. Conclusions

In this study, we collected data from multiple media platforms across English (Reddit) and Chinese (Toutiao, Sina). Our primary objective was to achieve high-performance controversy detection with minimal information, validated through “one-page” (20% hot comments) and early detection (3–6 h) experiments. By quantitatively evaluating feature discriminability in LR, RF, and XGBoost algorithms, we identified generalized features for controversy detection: the ascending gradient in likes/replies between superior and inferior comments proves to be an efficient and cross-lingual indicator. Crucially, conflicting posts exhibit significantly higher AG ratios, with >93.25% detection accuracy achieved using psychological features on Reddit (LightGBM/SVM). These novel AG/TAG psychological features demonstrated consistent algorithmic enhancement, boosting performance by >10% when added to baseline models and maintaining cross-platform robustness (90.21% on Toutiao, 83.66% on Sina). We interpret this through the backfire effect mechanism in ideological conflicts. For applications, even 20% hot comments (TAG-dominated) achieve 87.12% accuracy, with AG features gaining dominance beyond the critical 0.2 comment ratio threshold. Early detection using just 3–6 h of data reached 85.28–93.25% accuracy, proving our approach enables efficient controversy identification without complex semantic analysis or global network structures.

6. Future Work

This study has some limitations. Firstly, the algorithms used in this work are relatively basic, and we have not explored more advanced or diverse modeling approaches (e.g., deep learning architectures or ensemble strategies) that might further enhance performance or reveal deeper insights. Then, our current methodology primarily identifies controversial content based on the manifestation of conflict in user interactions. We have not, however, provided a direct causal correlation between the observed ascending gradient patterns and the emergence of conflict.

Despite these limitations, they point to several promising directions for future research. One key direction is to explore hybrid models that combine the advantages of our lightweight behavioral signals with the richness of content and structural information for more comprehensive controversy detection. Furthermore, future work could expand to more diverse platforms beyond Reddit and Toutiao to validate and refine our models, ensuring broader applicability. Investigating the underlying causal mechanisms that lead to the observed patterns, perhaps through controlled experimental designs, would also be a valuable direction to establish a direct causal link. Finally, while this study focuses on early controversy detection, expanding the temporal scope of analysis to understand the long-term evolution of controversies would offer deeper insights into online dynamics. These unresolved questions collectively challenge future computational sociology research, prompting us to explore how to solve problems in online media more quickly and effectively with limited data and computing power and whether psychological mechanisms can indeed yield local features more powerful than global ones.