A Human–AI Collaborative Pipeline for Decision Support in Urban Development Projects Based on Large-Scale Social Media Text Analysis

Abstract

The rapid growth of digital communication platforms has generated vast volumes of user-generated textual data and digital footprints, creating growing demand for scalable artificial intelligence systems capable of supporting evidence-based decision-making. This study proposes and evaluates a human–AI collaborative analytical pipeline for multi-class sentiment and aggression analysis of large-scale social media data (N = 15,064 messages) related to an urban infrastructure project. The proposed framework integrates standard NLP preprocessing, machine learning-based classifiers, temporal aggregation, and controlled large language model (LLM)-assisted classification within a structured analytical workflow that incorporates expert validation and oversight. A stratified manual validation procedure (n = 301) demonstrated substantial inter-annotator agreement (κ = 0.70) and stable multi-class classification accuracy (80%). The results indicate that combining sentiment polarity and aggression detection as complementary linguistic indicators improves sensitivity to shifts in discourse dynamics and enables early identification of emerging social tension. The study demonstrates the potential of human–AI collaborative analytical frameworks for transparent, interpretable, and predictive large-scale social media analysis in decision-support contexts.

1. Introduction

The rapid development of artificial intelligence (AI) technologies and their integration into nearly all areas of social and economic activity have significantly expanded the possibilities of data-driven decision-making. In this context, the study of human–AI collaboration has become increasingly important for addressing complex analytical tasks and supporting decision-making in data-intensive environments. Researchers increasingly conceptualize artificial intelligence not only as an automated tool but also as a collaborative partner capable of augmenting human expertise and improving analytical performance. Hybrid human–AI systems have demonstrated the potential to enhance the quality and efficiency of decision-making processes in situations where large volumes of heterogeneous information must be processed and interpreted [1,2,3].

The development of large language models (LLMs) has expanded the possibilities for integrating collaborative intelligence. In such configurations, AI systems provide computational and analytical support, while human experts ensure contextual interpretation, methodological control, and verification of results [4,5,6,7].

At the same time, the rapid development of digital communication platforms has led to an unprecedented growth in user-generated textual data in online environments. Social media platforms, forums, and comment sections continuously generate large volumes of unstructured text reflecting opinions, emotions, and patterns of social interaction. This transformation of the communication landscape has brought natural language processing (NLP) technologies to the forefront as key tools for extracting meaningful information from complex textual data, supporting analytical tasks in a wide range of domains, and enabling predictive analytics [8,9,10,11,12,13].

Despite the growing body of research on human–AI collaboration and AI-assisted text analytics, several challenges remain insufficiently addressed. In particular, existing studies often focus on individual analytical methods—such as sentiment analysis or LLM-based interpretation—rather than integrating multiple analytical components within a unified analytical framework. Moreover, relatively limited attention has been paid to combining sentiment analysis, aggression detection, temporal analysis, and LLM-assisted interpretation within a single analytical pipeline designed for large-scale analysis of social media data.

To address this gap, this study proposes and evaluates a human–AI collaborative analytical framework designed for large-scale analysis of social media content related to urban development projects. A distinctive feature of this approach is the integration of analytical methods and AI technologies into a single, human-controlled pipeline, as well as the combined use of traditional sentiment analysis methods, aggression detection, and semantic interpretation.

The LLM-based framework allows for the analysis of the dynamics of emotional and communicative indicators, identifying persistent communication patterns and early warning signs of negative changes in public perceptions of a situation.

The proposed framework is applied to social media data related to the implementation of a large urban infrastructure project. The empirical analysis demonstrates how AI-enhanced analytical models can support the structured interpretation of complex textual data, reveal communication patterns, and provide a basis for the predictive assessment of social tensions in digital communication environments.

The main contributions of this study are as follows:

(1)

Development of a human–AI collaborative analytical framework for large-scale analysis of social media data.

(2)

Integration of sentiment analysis, aggression detection, and LLM-assisted classification within a unified analytical pipeline.

(3)

Incorporation of temporal aggregation of emotional indicators, enabling the identification of dynamic communication patterns and early signals of changes in public perception.

(4)

Application of the framework to social media data related to a large urban infrastructure project.

(5)

Demonstration of the role of LLMs as components of human-supervised analytical systems.

The contribution of this study is therefore not the development of a new standalone AI model, but the design of an integrated human–AI collaborative analytical framework that combines multiple complementary methods within a unified workflow.

1.1. Theoretical Background

1.1.1. Human–AI Collaboration

Effective human–AI collaboration requires the integration of mechanisms that ensure transparency, interpretability, and human oversight [14,15,16,17,18].

Human–AI collaboration is increasingly applied across knowledge-intensive domains, including scientific research, organizational decision-making, and complex socio-technical systems. These studies emphasize the importance of integrated analytical frameworks that combine AI capabilities with human expertise in order to enable efficient analysis of large datasets and support evidence-based decision-making in complex environments [19,20,21].

1.1.2. Sentiment Analysis

Sentiment analysis has become one of the most widely used methods for analyzing user-generated content. Early foundational studies defined sentiment analysis as the computational identification and classification of opinions, attitudes, and emotions expressed in text [22,23]. With the increasing role of social media in public communication, sentiment analysis has been widely applied to large-scale and rapidly evolving datasets. Such methods enable the identification of patterns of public opinion [24], provide analytical insights for predictive analytics [25], and support the assessment of citizens’ perceptions in urban and social analytics studies [26].

1.1.3. Aggression Detection

However, sentiment polarity alone often provides only a limited understanding of communication dynamics in online environments. The identification of heightened emotional expressions—particularly aggressive or hostile language—represents an additional analytical dimension that can improve the interpretation of digital communication processes. In AI-based analytical systems, aggression detection is typically formulated as a classification task that can be addressed using both rule-based and data-driven approaches [27]. Recent studies also emphasize the importance of aggression detection for identifying harmful language and hostile interactions in online environments [28,29].

1.1.4. LLMs in Text Analytics

The rapid development of large language models has further advanced sentiment analysis and text analytics by enabling more context-sensitive and scalable interpretation of user-generated content [30,31]. Nevertheless, recent findings indicate that LLM-based analytical systems require integration into structured analytical workflows in order to ensure reliability, interpretability, and reproducibility of results. Expert supervision therefore remains essential for validating analytical outputs and preventing potential biases or misinterpretations.

Advances in artificial intelligence have significantly expanded the methodological toolkit for large-scale text analysis [32]. When applied within human-supervised analytical workflows, LLM-based systems can complement traditional methods by supporting semantic interpretation and addressing linguistic variability in large and heterogeneous datasets.

1.1.5. AI in Urban Decision Support

In parallel, the integration of AI technologies into decision-support systems is becoming increasingly important in urban governance and infrastructure development. Recent studies highlight the growing role of AI and LLM-based systems in urban planning and decision-support frameworks, where they can assist planners in analyzing large volumes of textual and spatial data and support participatory planning processes involving multiple stakeholders [33,34]. AI-driven decision-support systems have also demonstrated their potential to address complex urban development challenges by integrating heterogeneous data sources and stakeholder perspectives [35,36]. At the same time, scholars emphasize that the effective use of AI in urban governance requires human-centered and expert-supervised approaches to ensure transparency, interpretability, and responsible use of AI technologies [37,38].

2. Materials and Methods

2.1. Data Collection and Preprocessing

The research material consisted of data related to the implementation of a large urban development project in the road and transport system of a metropolitan area.

The study is based on large-scale textual data collected from open digital platforms, including social media, blogs, microblogs, video hosting services, forums, and online news comment sections. The dataset consists of 15,064 messages in Russian. Data collection was conducted using automated monitoring tools and covered the entire period of the infrastructure project’s implementation, which was divided into four analytically defined stages corresponding to key decision-making and implementation phases.

Data collection was performed using a keyword-based query strategy implemented through the Brand Analytics monitoring system (Brand Analytics, Moscow, https://brandanalytics.ru/, accessed on 7 March 2026). The query included project-related keywords, location names, and terms associated with the infrastructure project, allowing the identification of relevant discussions. To ensure data relevance, posts and comments not directly related to the project context were excluded during the initial filtering stage. Duplicate messages, repost chains without substantive textual content, and system-generated messages were removed to reduce noise and artificial amplification effects.

While the exact dataset cannot be publicly shared due to platform restrictions, the data collection procedure can be replicated using similar keyword queries within the Brand Analytics system.

The dataset was organized into four stages corresponding to key phases of the urban infrastructure project, enabling temporal analysis of communication dynamics.

The dataset consists of Russian-language user-generated content collected from major Russian digital platforms, including VKontakte, Odnoklassniki, and DZen (Yandex).

Given the linguistic characteristics of the Russian language, preprocessing included normalization of inflectional forms and handling of morphological variation. Special attention was paid to features of Russian-language social media communication, including informal spelling, abbreviations, and expressive markers typical for online discourse.

Standard preprocessing procedures also included tokenization, removal of non-textual elements, and filtering of noise introduced by reposts and automated content.

The analyzed dataset consists exclusively of publicly available user-generated content. No personal or confidential data were collected or processed. The resulting textual data were normalized using standard preprocessing procedures, including cleaning, token normalization, and noise reduction.

Following preprocessing, a control sample was subjected to expert evaluation for each project stage to validate the analytical results (see Table 1).

Table 1. Quantitative Data Characteristics.

Stage	Designation	Period	Number of Characters	Messages	Engagement	Loyalty	Audience
1	Urban Planning and Land Commission	1 January 2019–5 April 2019	1,122,626	475	3758	2, 3	720,051
2	Public hearings	6 April 2019–9 June 2019	5,539,515	3022	39,973	1, 4	19,171,007
3	First leg of construction	10 June 2019–31 December 2019	11,142,369	7906	167,545	4, 9	39,644,286
4	Second leg of construction	1 January 2020–30 April 2020	6,155,658	3661	85,205	1, 2	11,280,584

The “Loyalty” indicator presented in Table 1 is a metric provided by the Brand Analytics platform. It reflects the balance between positive and negative user reactions within the analyzed dataset and is calculated as the ratio of positive to negative sentiment messages over a given period.

2.2. Methods

2.2.1. Overall Analytical Framework

The data analysis was conducted using an approach described in [39]. Data collection was carried out using the Brand Analytics social media and media monitoring system. TextAnalyst 2.32 and GPT-based LLM system (internal GPT-5.2 configuration used within the OpenAI API environment) were employed for data analysis and interpretation, while Tableau was used for interactive data visualization and exploratory analytics. The LLM configuration used in this study corresponds to a more recent model compared to the GPT-3.5, GPT-4, and GPT-4o models applied in our previous work [39], which was conducted on a different dataset and addressed different research tasks.

The study was conducted in three stages: preliminary, main, and final.

An AI-enhanced NLP framework was applied for the analysis of textual data, specifically designed for scalable processing of social media content. This framework integrates traditional text analysis methods, machine learning-based classifiers, and LLM-assisted analysis into a unified and reproducible analytical pipeline under the supervision of a human expert.

2.2.2. Sentiment Analysis

The empirical dataset included large volumes of unstructured textual data and digital footprints analyzed using a combined sentiment analysis approach. The sentiment algorithm is based on statistical Conditional Random Fields (CRFs) and lexicon-based models. The sentiment index is analogous to the Net Sentiment metric widely used in social media analytics. Aspect-based sentiment analysis extends the baseline approach by identifying sentiment evaluations across subcategories. The system performs large-scale sentiment classification by assigning each message to one of three categories (positive, neutral, or negative), enabling efficient analysis of large textual datasets.

The use of automated sentiment analysis ensured scalable processing of large-scale textual data streams. For automated sentiment analysis of textual messages, an LLM-based system was additionally employed as an auxiliary analytical tool. The LLM-based classification was performed using a GPT-based LLM (GPT-5.2 configuration used within the OpenAI API environment). A fixed prompt template was applied across all datasets to ensure methodological consistency. The temperature parameter was set to 0 to ensure deterministic classification results. Each message was processed independently using a single inference call. The prompt explicitly specified the sentiment categories (positive, neutral, negative) and required the model to return only a categorical label (Appendix A).

The sentiment classification procedure is based on context-dependent semantic analysis performed by the LLM at the level of individual messages. Each text fragment is classified by considering lexical meaning, syntactic structure, and contextual relations, after which it is assigned to one of three sentiment categories (positive, negative, or neutral). Classification is performed relative to a predefined object of analysis (e.g., an infrastructure project or its specific components).

2.2.3. Aggression Detection

In addition to sentiment polarity detection, aggression detection was implemented as an independent classification task within the domain of human–AI interaction. Aggression was treated as a distinct communicative dimension reflecting heightened emotional intensity and hostile interaction patterns that may not be fully captured by sentiment analysis alone.

The novelty of the proposed approach lies in combining sentiment analysis with aggression analysis, which was conducted using TextAnalyst 2.32 [39]. This integration enables a more accurate assessment of actors’ emotional states and attitudes toward analyzed events, situations, or objects and supports predictive modeling. Aggression is treated as an independent communicative feature, distinct from overall sentiment polarity.

Aggression analysis was performed as a separate text classification task. The classification is based on lexical, syntactic, and contextual features indicative of aggressive communication, which were identified by experts and incorporated into the model. The model output is a categorical label corresponding to a three-level aggression scheme (no aggression, aggression, strong aggression). Aggressive content was further classified by intensity level, allowing differentiation between mild hostile expressions and explicit verbal attacks.

Automated aggression detection may be sensitive to contextual ambiguity, including irony, quotations, or reproduced offensive language. Accordingly, aggression labels reflect algorithmic assessments rather than definitive judgments of speaker intent. A subset of aggression classifications was manually validated to assess detection reliability.

2.3. Validation

The quality and reliability of the automatic clustering of sentiment and aggression were assessed through manual verification, which involved a stratified subsample of the dataset consisting of 301 messages. This sample corresponds to approximately 2% of the total corpus (N = 15,064). To avoid bias, the sample matched the volume of data from each stage, the number of digital platforms used by actors, message categories, and actor types (Stage 1—10 messages; Stage 2—60; Stage 3—158; Stage 4—73). This strategy was used to ensure accurate coverage of heterogeneous content and temporal dynamics. The number of messages selected for each stage reflects the relative size of that stage within the overall dataset. In particular, the Stage 1 subset (n = 10) corresponds to approximately 2% of the data available for that stage and is consistent with the overall sampling proportion applied across the dataset. Stage 1 results are excluded from reliability conclusions due to sample size.

Two independent annotators labeled the data using a three-class sentiment scheme (positive, neutral, negative) and a three-level aggression scheme (no aggression, aggression, strong aggression). Disagreements were resolved through discussion to reach consensus.

Inter-annotator agreement reached 85%. Cohen’s kappa (κ = 0.70) indicates substantial agreement beyond chance across multi-class categories.

Comparison of automated classification outputs with the consensus human labels yielded an overall average accuracy of 80%, demonstrating stable performance in multi-class sentiment and aggression detection. The highest performance was observed for the “neutral” and “no aggression” classes (F1 > 0.85), while lower F1-scores were obtained for less frequent classes (“strong aggression” and “positive”), with values of 0.60 and 0.55 respectively, which is typical for imbalanced datasets.

These results confirm the methodological robustness of the proposed AI-enhanced NLP framework and justify the use of aggregated indicators for temporal and predictive analysis.

2.4. Indices of Social Stress and Well-Being

While sentiment analysis reflects the overall emotional polarity of text, aggression analysis provides complementary insights into communicative intensity and conflict dynamics. Together, these dimensions enable a more nuanced analysis of large-scale textual data streams. The emotional states of actors in the context of event development are further illustrated by social stress and well-being indices, calculated using the methodology described in [27,39].

The social stress index and the social well-being index are composite indicators derived from linguistic markers identified in the analyzed textual data. The indices are calculated based on the distribution and intensity of semantically significant markers extracted from negative and positive clusters of user-generated content.

The calculation procedure includes several stages. First, sentiment analysis and semantic clustering are performed to identify negative and positive content groups. Second, key linguistic markers are extracted from each cluster using semantic network and associative analysis. Markers from the negative cluster are used to construct the social stress index, while markers from the positive cluster form the basis of the social well-being index.

Each marker is assigned a weight reflecting its relative importance and a rank corresponding to its frequency and prominence in the dataset. The indices are then calculated as normalized weighted sums of these markers, taking into account both their frequency and semantic significance. Formally, the index can be represented as a weighted aggregation of marker contributions:

Index = Σ (R_i × B_i)/(R_max × B_max)

where R_i denotes the rank of marker i, B_i represents its weight, and R_max and B_max correspond to the maximum observed values of rank and weight, respectively.

For the social stress index, markers associated with negative sentiment and aggressive communication are assigned negative polarity, whereas for the social well-being index, markers associated with positive sentiment and supportive communication are assigned positive polarity.

This approach allows the indices to capture both the intensity and the semantic structure of communication, providing a more nuanced representation of public perception dynamics compared to standard sentiment metrics.

To support predictive analytics, the study operationalized an early warning indicator of emerging social tension based on the temporal dynamics of aggression-related content. The indicator was defined as a sustained increase in the relative share of aggressive messages within a rolling temporal window preceding periods of peak discussion intensity. The rolling window was defined at each stage of the project as a fixed-length time interval. This procedure, combined with semantic and associative network analysis, allows for the identification of implicit markers of negative escalation before explicit indicators of changes in public sentiment emerge.

Temporal aggregation was performed using a rolling window approach with a fixed-length time interval of 7 days. This window size was selected to balance sensitivity to short-term fluctuations with robustness to noise in user-generated content.

For the purposes of early warning detection, the local baseline was defined as the average relative share of aggressive messages across the two preceding rolling windows. A warning signal was recorded when the relative share of aggressive messages exceeded this baseline in two consecutive rolling windows.

This rule was applied consistently across all stages of the analysis and was used to identify sustained increases in aggression-related communication that may indicate emerging social tension. Such increases were interpreted as potential signals of growing communication intensity or emerging social tension.

The reproducibility of the study’s methodology is determined by the fact that the developed analytical pipeline was tested with a sequential implementation of the data collection, analysis, and interpretation stages: data collection, preprocessing, sentiment classification, aggression detection, and temporal aggregation. The LLM-based component used a fixed prompt template and deterministic output parameters. All analytical procedures were conducted using publicly available data and widely used analytical platforms, allowing the analytical workflow to be replicated in similar research contexts.

2.5. Study Design

The design of the preliminary stage is presented in Figure 1.

Figure 1. Workflow of the preliminary stage of the study, including data collection, preprocessing, and initial exploratory analysis.

The procedures for the main phase of the study are shown in Figure 2.

Figure 2. Analytical procedures applied during the main stage, including sentiment analysis, aggression detection, and temporal aggregation.

The algorithm of actions at the final stage of the study is shown in Figure 3.

Figure 3. Final stage workflow, including interpretation of results and identification of key communication patterns.

The overall architecture of the human–AI collaborative analytical pipeline used in the study is presented in Figure 4.

Figure 4. Architecture of the human–AI collaborative analytical pipeline integrating NLP methods, LLM-based analysis, and expert supervision.

2.6. Ethical Considerations

The use of large language model-based tools in this study complied with ethical research standards. Generative AI (GenAI) tools were applied exclusively for auxiliary analytical purposes to support sentiment and aggression classification, as well as the interpretation and visualization of aggregated results, within a predefined methodological framework developed and controlled by the authors. These tools did not generate original research data, formulate research hypotheses, or autonomously produce scientific conclusions. All analytical procedures and interpretations were subject to human oversight, and a subset of automatically classified materials was manually reviewed to assess reliability. The use of GenAI involved only publicly available content and did not replace human judgment in data interpretation or scientific decision making.

3. Results

Empirical analysis revealed three key patterns in the dynamics of social media discourse surrounding the project. First, discussion activity was concentrated on a limited number of digital platforms, with social media serving as the primary arena of communication. Second, sentiment dynamics demonstrated a gradual shift from predominantly positive reactions in the early stages to a more neutral and conflict-sensitive discourse in later stages. Third, indicators related to aggression consistently preceded increases in discussion intensity and negative sentiment, suggesting their potential role as early warning signs of rising social tension.

3.1. Preliminary Stage of the Study

The preliminary stage of the study involved data collection and cleansing, analysis of source types, selection and ranking of preferred sources by actors generating content and discussing issues related to the project, audience re-search, and engagement analysis.

Data analysis at the preliminary stage showed that social media activity predominates at all stages. In the first stage, a small amount of content is also present in blogs and microblogs (Figure 4). In the second, third, and fourth stages, video hosting content appears, ranking second in audience and engagement, while microblogs and messengers rank third and fourth, respectively. Blogs and forums show insignificant engagement levels (Table 2, Table 3, Table 4 and Table 5).

Table 2. Distribution of message sources across digital platforms during Stage 1, showing the dominance of social networks in early-stage communication.

Source Type	Audience	Engagement
Social networks	889,217	3758
Blogs	5019	0
Microblogs	304	0

Table 3. Distribution of message sources across digital platforms during Stage 2, reflecting increased activity during the public discussion phase.

Source Type	Audience	Engagement
Social networks	24,297,061	38,686
Video	9,676,124	733
Messengers	128,153	0
Microblogs	40,113	11
Blogs	15,623	65

Table 4. Distribution of message sources across digital platforms during Stage 3, indicating increased activity on video platforms during the construction phase.

Source Type	Audience	Engagement
Social networks	71,762,164	165,082
Video	26,555,256	671
Microblogs	3,225,640	207
Messengers	2,165,294	0
Blogs	409,459	866
Forums	—	719

Table 5. Distribution of message sources across digital platforms during Stage 4, reflecting a decline in communication intensity and diversification of content sources.

Source Type	Audience	Engagement
Social networks	17,871,601	82,681
Video	1,549,558	1688
Microblogs	1,310,369	26
Messengers	246,656	0
Blogs	151,456	158
Forums	—	652

Throughout all stages of the study, the social network VKontakte led among digital platforms. The subsequent positions varied: in the first stage, Facebook, Instagram, and Odnoklassniki ranked second, third, and fourth, respectively (Table 6). In the second stage, YouTube, Facebook, and Instagram followed (Table 7). In the third stage, YouTube, Odnoklassniki, and Instagram were the next most popular (Table 8). In the fourth stage, Facebook, Instagram, and YouTube occupied the subsequent positions (Table 9).

Table 6. Ranking of digital platforms by message volume during Stage 1, showing the dominance of VKontakte and other major social media platforms.

Platform	Number of Messages
http://www.vk.com	703,218
http://www.facebook.com	92,976
http://www.instagram.com	79,766
http://www.ok.ru	13,257
http://www.livejournal.com	4807
http://www.twitter.com	304
http://www.zen.yandex.ru	212

Table 7. Ranking of digital platforms during Stage 2, reflecting increased engagement on video-sharing platforms.

Platform	Number of Messages
http://www.vk.com	13,502,541
http://www.youtube.com	9,676,124
http://www.facebook.com	7,239,519
http://www.instagram.com	1,803,672
http://www.ok.ru	1,727,876
http://www.telegram.org	128,153
http://www.twitter.com	40,113
http://www.my.mail.ru	23,453
http://www.livejournal.com	13,279
http://www.zen.yandex.ru	1862
http://www.liveinternet.ru	482

Table 8. Ranking of digital platforms during Stage 3, indicating the growing importance of video and social media platforms during the construction phase.

Platform	Number of Messages
http://www.vk.com	40,550,886
http://www.youtube.com	26,555,256
http://www.ok.ru	11,847,956
http://www.instagram.com	9,905,314
http://www.facebook.com	9,417,752
http://www.twitter.com	3,225,640
http://www.telegram.org	2,165,294
http://www.zen.yandex.ru	353,089
http://www.livejournal.com	54,776
http://www.my.mail.ru	40,256
http://www.liveinternet.ru	1594

Table 9. Ranking of digital platforms during Stage 4, showing changes in platform activity during declining public engagement.

Platform	Number of Messages
http://www.vk.com	10,920,904
http://www.facebook.com	3,540,534
http://www.instagram.com	2,999,520
http://www.youtube.com	1,549,558
http://www.twitter.com	1,310,369
http://www.ok.ru	410,643
http://www.telegram.org	246,656
http://www.zen.yandex.ru	122,482
http://www.livejournal.com	28,974

3.2. Main Stage of the Study

3.2.1. Sentiment Analysis

An analysis of the digital platforms on which relevant content was identified shows that the largest volume of messages was generated on social media platforms. During Stages 1 and 2, sentiment distributions in social media were dominated by a positive cluster, whereas during Stages 3 and 4, neutral sentiment prevailed. At Stage 1, a small proportion of neutral messages was also observed in blogs and microblogs.

At subsequent stages, video hosting platforms emerged as the second most significant source of content.

Social media served as the primary arena for positive and neutral reactions, while video-sharing platforms and microblogs demonstrated a higher concentration of negative sentiment in later stages of the project. These patterns highlight the importance of considering both temporal dynamics and platform heterogeneity when interpreting sentiment trends in large social media datasets (Table 10, Table 11, Table 12 and Table 13).

Table 10. Distribution of sentiment categories across data sources during Stage 1.

Source Type	Negative	Neutral	Positive
Social networks	74,372	393,516	421,329
Blogs	—	5019	—
Microblogs	—	304	—

Table 11. Distribution of sentiment categories across data sources during Stage 2.

Source Type	Negative	Neutral	Positive
Social networks	10,089,657	8,260,203	5,947,201
Video	6,202,908	3,464,659	8557
Messengers	10,540	62,921	54,692
Microblogs	6017	5028	29,068
Blogs	5198	6007	4418

Table 12. Distribution of sentiment categories across data sources during Stage 3.

Source Type	Negative	Neutral	Positive
Social networks	9,228,288	32,358,272	30,175,604
Video	6,543,102	2,956,366	10,755,788
Microblogs	775,372	1,843,682	606,586
Messengers	254,173	1,462,375	448,746
Blogs	38,056	277,641	93,762

Table 13. Distribution of sentiment categories across data sources during Stage 4.

Source Type	Neutral	Positive	Negative
Social networks	11,612,580	4,668,844	1,590,177
Video	494,546	424,240	630,772
Microblogs	62,769	611	1,246,989
Messengers	199,619	11,361	35,676
Blogs	128,011	14,258	9187

However, it should be noted that a substantial share of the analyzed content was generated by official sources, affiliated journalists, public relations actors, and media organizations. This type of content predominantly forms neutral and positive sentiment clusters and, to a large extent, does not directly reflect the reactions of local residents. In particular, this pattern is evidenced by the predominance of neutral and positive content on the VKontakte platform.

Audience and engagement analysis indicates that negative reactions exhibit the lowest values across all stages. Positive audience reactions prevail during Stages 1 and 2, while positive engagement metrics are most pronounced during Stages 2 and 3. The neutral audience cluster dominates during Stages 3 and 4, whereas neutral engagement is most prominent at Stages 1 and 4 (Table 14, Table 15, Table 16 and Table 17).

Table 14. Audience sentiment and engagement levels during Stage 1, showing higher positive engagement compared to negative reactions.

Audience		Engagement
Sentiment	Number of Messages	Sentiment	Value
Positive	421,329	Neutral	1466
Neutral	398,839	Positive	1388
Negative	74,372	Negative	904

Table 15. Audience sentiment and engagement levels during Stage 2, reflecting increased overall activity and engagement intensity.

Audience		Engagement
Sentiment	Number of Messages	Sentiment	Value
Positive	16,314,320	Positive	16,925
Neutral	11,798,818	Neutral	13,774
Negative	6,043,936	Negative	9274

Table 16. Audience sentiment and engagement levels during Stage 3, indicating a dominance of neutral audience responses.

Audience		Engagement
Sentiment	Number of Messages	Sentiment	Value
Positive	42,080,486	Positive	83,656
Neutral	45,198,336	Neutral	73,035
Negative	16,838,991	Negative	10,854

Table 17. Audience sentiment and engagement levels during Stage 4, showing reduced engagement and stabilization of sentiment.

Audience		Engagement
Sentiment	Number of Messages	Sentiment	Value
Positive	5,119,314	Positive	20,693
Neutral	12,497,525	Neutral	54,898
Negative	3,512,801	Negative	9614

An analysis of activity across different actor types shows that during Stage 1, community accounts were characterized by the most active formation of a positive sentiment cluster. In contrast, personal accounts, unspecified accounts, and media accounts exhibited substantially lower levels of positive sentiment. Negative sentiment demonstrated the lowest values across all actor types at this stage (Table 18, Table 19, Table 20 and Table 21).

Table 18. Sentiment distribution across actor types during Stage 1, highlighting the dominant role of community accounts in generating positive content.

Actor Type	Positive	Neutral	Negative
Communities	370,509	269,188	41,754
Personal Profile	43,979	64,866	0
Undefined	6841	0	1365
Media Account	0	726	0

Table 19. Sentiment distribution across actor types during Stage 2, reflecting increased diversity of sentiment contributions.

Actor Type	Positive	Neutral	Negative
Personal Profile	9,498,797	2,737,192	2,656,878
Communities	5,819,756	4,065,929	0
Media Account	573,609	0	1,195,629
Undefined	0	1,005,547	0

Table 20. Sentiment distribution across actor types during Stage 3, indicating a shift toward neutral and mixed sentiment patterns.

Actor Type	Positive	Neutral	Negative
Communities	16,624,785	17,373,717	5,549,986
Personal Profile	9,560,208	0	0
Media Account	13,868,007	9,412,623	7,161,780
Undefined	2,027,486	2,292,284	430,205

Table 21. Sentiment distribution across actor types during Stage 4, showing reduced activity and more balanced sentiment distribution.

Actor Type	Positive	Neutral	Negative
Communities	2,869,830	6,873,154	0
Personal Profile	1,532,846	4,041,750	2,535,162
Media Account	0	0	247
Undefined	474,807	1,187,148	664,249

The analysis of sentiment across different message types may indicate the dynamics of situation development and changes in actors’ perceptions. The data suggest that negative reactions initially appeared during Stage 1 primarily in reposts without additional commentary (47,586), as well as in posts, reposts with commentary, and comments. During Stage 2, the number of negative reactions increased substantially in posts (3,308,532), as well as in comments and reposts both with and without additional text. At Stage 3, a further increase in negative reactions was observed, particularly in posts (12,430,094), comments, and reposts with and without commentary. At Stage 4, a sharp decline in all types of reactions was recorded, including negative reactions, with a decrease observed in comments (2,393,219), posts, and reposts both with and without additional content.

Digital footprints provide the most reliable basis for assessing residents’ reactions to the development of the situation surrounding the project. At the first stage, comments and views were characterized by predominantly positive sentiment, likes were mainly neutral, while reposts already demonstrated a predominance of negative reactions. During the second stage, a surge in negative comments and reposts was observed, driven by residents’ active involvement in discussions related to the project’s implementation, alongside an increase in positive reactions expressed through views and likes. At the third stage, the level of negative sentiment decreased, while positive and neutral sentiments became more prevalent. The fourth stage was marked by a general decline in actors’ interest and a predominance of neutral reactions, with the exception of a substantial number of negative reposts (Table 22).

Table 22. Sentiment distribution across digital footprints (likes, comments, reposts, and views), highlighting differences in user engagement behavior.

Stage 1
Metric	Positive	Neutral	Negative
Comments	292	224	148
Likes	1055	1216	596
Reposts	41	26	160
Views	54,141	35,196	9790
Stage 2
Metric	Negative	Neutral	Positive
Comments	2891	2614	1887
Likes	5389	10,493	14,347
Reposts	994	667	691
Views	152,066	220,994	398,790
Stage 3
Metric	Neutral	Positive	Negative
Comments	6520	4151	1880
Likes	65,665	77,530	8425
Reposts	850	1975	549
Views	952,442	1,086,683	246,921
Stage 4
Metric	Neutral	Negative	Positive
Comments	3128	865	744
Likes	51,279	8204	19,774
Reposts	491	545	175
Views	326,082	44,894	146,121

In the main stage of the study, rankings were also formed for actors generating negative and positive content, as well as for content with negative and positive sentiment based on audience size and engagement.

3.2.2. Analysis of Aggression

In the main stage of the study, an analysis of aggression was conducted, which serves as an accurate marker of emotional tension or its absence when discussing a specific situation. A three-point scale was used to determine aggression (absence of aggression–presence of aggression–strong aggression).

The manifestations of aggression in the audience and engagement metrics are shown in Table 23.

Table 23. Audience aggression and engagement levels, showing the relationship between aggression intensity and user interaction.

Stage	No Aggression	Aggression	Strong Aggression
Stage 1	3758	0	0
Stage 2	167,431	108	6
Stage 3	39,864	103	6
Stage 4	85,204	0	1

The dynamics of aggression in text messages and digital footprints of actors was also studied (Table 24).

Table 24. Dynamics of aggression in digital footprints, indicating temporal patterns of communicative tension.

Stage 1
Metric		No Aggression		Aggression
Comments		664		0
Likes		2867		0
Reposts		227		0
Views		99,127		0
Stage 2
Metric	No Aggression		Aggression	Strong Aggression
Comments	7369		20	3
Likes	30,154		72	3
Reposts	2341		11	0
Views	771,606		244	0
Stage 3
Metric	No Aggression		Aggression	Strong Aggression
Comments	12,530		21	0
Likes	151,529		87	4
Reposts	3572		0	2
Views	2,285,583		3	460
Stage 4
Metric	No Aggression		Aggression	Strong Aggression
Comments	4736		1	0
Likes	79,257		0	0
Reposts	1211		0	0
Views	517,097		0	0

The presence of aggression at all stages of the study, especially the recording of strong aggression in the second, third, and fourth stages, indicates heightened emotional tension and negative perceptions associated with the project discussion.

Empirical analysis shows that peaks in aggression-related indicators consistently preceded or coincided with subsequent increases in overall discussion volume and negative sentiment, particularly during the transition from Stage 1 to Stage 2. This temporal ordering suggests that aggression dynamics may function as an early signal of intensifying public tension, emerging before broader sentiment polarization becomes visible at the aggregate level.

This interpretation is supported by the analysis of the core associative network. During thematic modeling and the identification of the semantic network, stimuli for associative search were identified. After constructing the semantic network, a semantic core with connection weights of 100-99 was extracted, from which a list of stimuli was formed through expert analysis. The semantic network revealed the explicit semantic accents expressed by actors when discussing project-related issues. However, the most valuable insights are the implicit opinions and assessments identified through the associative network analysis. An associative search was conducted, and reactions within the negative and positive clusters were analyzed separately.

3.2.3. Indices of Social Stress and Well-Being

The dynamics of public perception of the project’s implementation reveal the following trends.

(1)

User interest peaks during the transition from stage 1 to stage 2, when a sharp increase in both negative and positive reactions is observed. Strong aggression is recorded, which is a marker of escalating emotional reactions among residents.

(2)

Stage 4 is characterized by a sharp decline in stakeholders’ interest in the project, driven by a shift in attention to the spread of the pandemic. This decline in audience engagement is accompanied by an increase in negative reactions, leading to the formation of negative stereotypes and persistent behavior patterns when evaluating the project’s implementation, especially in the disputed area, which naturally leads to a decline in the social well-being index.

(3)

The study results show that the increase in negative responses from stakeholders indicates a potential conflict situation, which, in the absence of external social factors distracting residents, could have led to increased social tension around the project’s implementation. Such dynamics might have included a broader expression of dissatisfaction in digital communication spaces, with potential implications for offline urban contexts.

In addition, during the main stage of the study, indices of social stress and well-being were identified for each stage of the project’s implementation, and their temporal dynamics were analyzed (Table 25).

Table 25. Temporal dynamics of the social stress and social well-being indices, illustrating changes in public perception across project stages.

Stage	Social Stress Index	Social Well-Being Index
Stage 1	0.800	4.300
Stage 2	5.500	3.900
Stage 3	4.050	3.400
Stage 4	2.050	3.100

Table 25. Temporal dynamics of the social stress and social well-being indices, illustrating changes in public perception across project stages.

Stage	Social Stress Index	Social Well-Being Index
Stage 1	0.800	4.300
Stage 2	5.500	3.900
Stage 3	4.050	3.400
Stage 4	2.050	3.100

Calculation of the social stress index and the social well-being index was conducted across the entire dataset [27]. Indicators of social well-being are determined by the increase in the amount of neutral and positive content generated by official structures.

Moreover, it should be noted that the conflict zone is only one section of the project. The results of the analysis of residents’ perception of the problem, the level of tension, are blurred in the overall database, which does not diminish the urgency of the issue. On the contrary, the fixation of the problem in a specific section of the road transport project within a large dataset demonstrates its urgency. It is evident that a sample limited to the boundaries of the specified section would show an extremely high level of conflict.

3.3. Final Stage of the Study

During the final stage of the research, a list of grievances that elicited the strongest negative reactions from residents was compiled, as well as a list of topics associated with social approval of the project.

The active informational support of the project led to a large amount of content in the aggregate database being generated by official sources, affiliated journalistic, PR, media, and other structures. This content predominantly constituted neutral and positive clusters. The results of the residents’ negative perception of this issue are blurred in a large amount of digital material dedicated to the overall project implementation.

Additionally, the study revealed that the conflict digital zone arises only around one section of the project, and the negative attitude of the stakeholders is then analogously transferred to the entire project.

3.3.1. Semantic Accents of the Negative Cluster

The negative cluster is defined with fears and concerns about the risks associated with the project’s implementation: construction work could damage residential buildings located in the immediate vicinity of the construction site; the expected relocation of residents to other areas following the demolition of buildings that are in good condition but impede construction; the construction will cause a traffic jam; and the poor planning of the new line. Additional concerns include the destruction of infrastructure in the existing residential area, the erosion of the historic urban environment, and the anticipated deterioration of residents’ quality of life.

Negative sentiments extend beyond criticism of the project and extend to all activities of the city administration, and also include suspicions of criminal practices by construction companies, often without specification of particular offenses.

A small number of topics included doubts about the veracity of the public hearing results, suspicions of forged resident signatures, a coalition with opponents of other urban development projects, distrust of official media, and concerns about the destruction and unjustified expansion of the traditional boundaries of the old city.

3.3.2. Semantic Accents of the Positive Cluster

The positive cluster compares topics describing the project’s implementation. Specifically, positive assessments of the various construction stages, the pace of construction, which is in line with the plan, and the advanced technologies and construction methods used by the contractors.

Important topics in the positive discourse include the development of the city’s transportation system, as well as plans for the improvement of urban space after construction is completed, landscaping of construction areas, and the restoration and beautification of the urban landscape.

In addition, favorable evaluations are expressed toward the activities of representatives of specific political parties who are perceived by residents as effectively defending their interests in the context of the project.

3.3.3. Key Findings

The results show that negative reactions concentrated around a specific project segment were extrapolated to the entire infrastructure project in social media discourse, supporting the hypothesis regarding the amplification of localized conflicts in digital environments.

The dynamics of project implementation perception reveal the following trends: during the Urban Planning and Land Commission stage, residents showed minimal interest in the project, and the content of this period consisted of few official announcements; user interest emerges at the junction between the first and second stages, marked by a sharp increase in reactions triggered by public hearings dedicated to the project; additionally, strong aggression arises at the second stage, reaching peak levels at the third stage (initial construction phase) and slightly declining at the fourth stage (continuation of construction); the fourth stage is characterized by a sharp decrease in interest in the topic, as society’s attention shifted to the onset of the COVID-19 pandemic. However, negative user reactions are noted to increase while overall activity towards the project decreases.

This scenario indicates a sustained, entrenched negative attitude of stakeholders towards the project; only one section of the project serves as a conflict zone, and the negative perception of it is extrapolated to the entire project.

Thus, the research results indicate a relatively high degree of social tension surrounding the project during the analyzed period. Data analysis showed that the emergence and rapid spread of the coronavirus may have contributed to a reduction in public attention toward the project against the construction of the disputed site, the negative association with which could have impacted the entire project. The obtained data also suggest the potential expansion of conflict-related public activity in both digital and urban contexts. In the short term, such activities might have re-emerged if external conditions had normalized within a limited time frame. In contrast, prolonged periods of heightened societal uncertainty contributed to a reduction in negative perceptions of the project’s implementation, as public attention shifted toward other dominant social concerns.

Based on the research findings, a forecast of the situation surrounding the project implementation was presented, potential conflict-prone areas were identified, and recommendations were developed to prevent the development of social tension and reduce the level of social stress, thereby increasing the chances of a favorable development of the situation in the perception of the residents.

From a predictive perspective, the results indicate that aggression-based indicators may serve as early warning signals of emerging social tension, complementing traditional sentiment polarity measures.

4. Discussion

From the perspective of the effectiveness of human–AI collaboration, the results of this study highlight the value of combining multiple linguistic dimensions—sentiment polarity, aggression, and temporal dynamics—within a single framework. While sentiment analysis captures general emotional orientation, aggression functions as an independent communicative signal reflecting heightened emotional intensity that may not be fully captured by sentiment polarity alone.

The use of LLM capabilities significantly expands the analytical potential of the NLP framework. In this case study, LLMs served as auxiliary clustering and classification tools, applied within the developed procedures with active and guiding human intervention. Strict control allowed for the use of LLMs’ contextual sensitivity and the maintenance of procedural transparency and methodological rigor. Data analysis demonstrated that mood and aggression classification using LLMs effectively complements machine learning-based classifiers when processing linguistic data obtained from heterogeneous and informal social media content.

More broadly, the results underscore the importance of framework-oriented approaches in AI-assisted text analysis. In large-scale social media contexts, the integration, scalability, and interpretability of heterogeneous methods are often more critical than optimizing individual models. While the framework was evaluated on a single empirical case and remains sensitive to linguistic ambiguity, it is modular and adaptable to other domains. Future research may extend the framework to multimodal data and further formalize temporal linguistic indicators for early signal detection.

In this context, the contribution of the study should be understood as a framework-oriented methodological advancement, focusing on the integration of heterogeneous analytical components rather than the development of a novel algorithm.

From a theoretical perspective, this study contributes to the emerging field of human–AI joint intelligence by demonstrating how multiple linguistic indicators can be integrated into a modular analytics pipeline. The results support the notion that hybrid analytics architectures combining automated natural language processing methods and human supervision provide a robust foundation for interpreting complex social media discourse.

Overall, the study demonstrates that AI-enhanced NLP frameworks integrating sentiment analysis, aggression detection, temporal modeling, and controlled LLM use provide a practical and methodologically sound approach to large-scale text analysis in applied settings.

The objective of the proposed approach is not to benchmark individual machine learning models, but to integrate complementary analytical components within a human-supervised analytical framework.

These results also suggest that human–AI collaboration pipelines may represent an effective architectural paradigm for integrating heterogeneous analytical methods in large-scale socio-technical data processing environments.

5. Practical Implications

The proposed human–AI collaborative analytical framework has direct practical relevance for decision-support systems in urban development. By enabling large-scale analysis of social media discourse, the framework allows stakeholders to monitor public perception dynamics and identify communication patterns associated with emerging social tension.

The results of the empirical analysis demonstrate that aggression-related indicators may serve as early warning signals, preceding increases in discussion intensity and negative sentiment. This provides an opportunity for proactive intervention and more effective management of communication processes in complex urban development contexts.

The framework can be applied by urban planners, policymakers, and communication specialists to identify conflict-prone areas, improve stakeholder engagement strategies, and support evidence-based decision-making. The integration of multiple analytical components—sentiment analysis, aggression detection, and temporal modeling—enables a more comprehensive understanding of digital communication dynamics compared to traditional approaches.

The practical application of the proposed framework to social media data related to the implementation of a major urban infrastructure project demonstrates the practical significance of this approach. In such systems, AI technologies ensure scalability of analysis and pattern identification, while human expertise guarantees contextual interpretation, methodological correctness, and the reliability of analytical conclusions. From a methodological perspective, the proposed framework demonstrates the value of integrating multiple analytical components into a single structure for large-scale text analysis. This approach facilitates the development of human-centered artificial intelligence systems focused on the transparency, interpretability, and reproducibility of analytical procedures.

The proposed approach therefore supports the development of human-centered decision-support systems in complex socio-technical environments.

6. Conclusions

This study developed and tested a collaborative human–AI analytical framework for large-scale analysis of user-generated data from social media. The proposed framework combines sentiment analysis, aggression detection, temporal aggregation, and classification with support for LLMs into a single analytical pipeline, operated under expert supervision.

Combining the computational capabilities of artificial intelligence with human expertise enables both descriptive and predictive analysis of communication dynamics in the digital environment.

The results of the study demonstrate that integrating traditional natural language processing methods with the capabilities of LLMs within human-controlled analytical procedures significantly expands the capabilities of analyzing complex textual datasets. In particular, the combined use of sentiment analysis and aggression detection enables the capture of both the general emotional polarity of discussions and more specific manifestations of communicative tension in online discussions.

The inclusion of temporal aggregation also makes it possible to identify dynamic changes in communication patterns and public sentiment over time.

Thus, the study contributes to the development of research related to the joint intelligence of humans and AI and highlights the potential of large-scale language model technologies as components of supervised and interpretable analytical systems.

At the same time, several limitations of the study should be noted. The empirical analysis was conducted using social media data, which represents only one source of information on public opinion and does not always fully reflect the diversity of positions and assessments existing in society.

Future research could aim to develop the proposed framework by expanding the set of analytical indicators, integrating multimodal data sources, and applying this approach in various social, organizational, and urban contexts. A promising direction is also the integration of explainable artificial intelligence methods and more complex LLMs to improve the transparency, interpretability, and methodological reliability of AI-based analytical procedures.

Overall, the study’s results highlight the growing importance of collaborative human–AI approaches for large-scale analysis of digital communications. By demonstrating how AI technologies and human expertise can be effectively integrated into structured analytical pipelines, this work contributes to the development of robust, interpretable, and practically applicable AI-based decision support systems in complex socio-technical systems.

The proposed approach highlights the value of integrative, human-supervised analytical pipelines as an alternative to purely model-centric solutions in AI-based text analysis.

Future research may include systematic benchmarking of individual components against alternative machine learning models.