AI-Driven Framework for Evaluating Climate Misinformation and Data Quality on Social Media

Abstract

In the digital age, climate change content on social media is frequently distorted by misinformation, driven by unrestricted content sharing and monetization incentives. This paper proposes a novel AI-based framework to evaluate the data quality of climate-related discourse across platforms like Twitter and YouTube. Data quality is defined using key dimensions of credibility, accuracy, relevance, and sentiment polarity, and a pipeline is developed using transformer-based NLP models, sentiment classifiers, and misinformation detection algorithms. The system processes user-generated content to detect sentiment drift, engagement patterns, and trustworthiness scores. Datasets were collected from three major platforms, encompassing over 1 million posts between 2018 and 2024. Evaluation metrics such as precision, recall, F1-score, and AUC were used to assess model performance. Results demonstrate a 9.2% improvement in misinformation filtering and 11.4% enhancement in content credibility detection compared to baseline models. These findings provide actionable insights for researchers, media outlets, and policymakers aiming to improve climate communication and reduce content-driven polarization on social platforms.

1. Introduction

Climate change content on social media has faced significant challenges in recent years [1]. One major concern is the ease with which misinformation spreads on these platforms, often without requiring user identity verification. For example, a 2022 study by Climate Action Against Disinformation reported that over 70% of climate misinformation posts analyzed on major platforms remained unflagged. Additionally, the monetization mechanisms on platforms like YouTube and Facebook incentivize the sharing of sensationalist or misleading information to attract engagement [2,3]. This flood of diverse content makes it difficult for users to distinguish between credible and unreliable sources. The lack of built-in quality control mechanisms has resulted in widespread information pollution [4]. Moreover, climate change discussions on social media often exacerbate polarization, fostering conflict rather than constructive discourse [5]. These factors make it increasingly difficult for the public to access accurate climate information and participate in informed decision-making. In response, several platforms have begun implementing policies to mitigate climate misinformation. Google, for instance, banned monetization of climate denial content in 2021. TikTok followed in 2023 with climate-specific content moderation guidelines, while Pinterest banned climate misinformation in both ads and user content by 2022 [6]. However, civil society and academic evaluations have revealed that enforcement remains inconsistent and largely ineffective. These gaps highlight the need for a more systematic, AI-based approach to evaluating content quality on social media. In this research [7], data quality is defined based on four key dimensions: (1) accuracy (content correctness), (2) credibility (source reliability), (3) completeness (availability of necessary information), and (4) relevance (alignment with the intended informational need). This approach integrates these dimensions into an automated evaluation system designed to detect, score, and filter climate-related content on social media. To address the identified issues, this study proposes a comprehensive AI-driven framework for evaluating and enhancing the quality of climate-related content shared on social platforms. The proposed system also integrates user feedback analysis and trend forecasting to ensure adaptability to evolving misinformation patterns.

The main contributions of this research are as follows:

• Proposal of a novel multi-stage classification pipeline: integrating sentiment-aware and domain-specific language models to identify and score the quality of climate-related content across platforms.
• Development of a content credibility scoring metric: combining linguistic features, source reliability, and user feedback signals into a unified quality index.
• Building a cross-platform dataset: composed of labeled social media posts (Twitter, YouTube, Reddit) with expert-reviewed quality tags, spanning diverse climate-related topics and misinformation themes.
• Evaluation of system performance: using metrics such as precision, recall, F1-score, AUC, and Brier score, and benchmarked against state-of-the-art misinformation classifiers.
• Conduct of ablation studies and bias tests: to assess robustness across domains, user demographics, and misinformation types, validating generalizability under real-world conditions.

Operational Definition and Evaluation of Data Quality

In this research, data quality refers to the reliability, credibility, and relevance of user-generated climate change content on social media platforms. Drawing on the ISO/IEC 25012 standard [8] and adapting it to the unique characteristics of social media data, the focus is on four core dimensions:

• Accuracy: The extent to which social media content reflects scientifically validated facts. Accuracy is verified through natural language inference techniques and content alignment with trusted climate science repositories, such as IPCC and NASA reports.
• Credibility: Credibility is assessed by examining metadata such as user profile verification, posting history, domain expertise, and network behavior. A trust scoring algorithm, enhanced by graph-based modeling, evaluates the source’s historical reliability.
• Completeness: Posts are analyzed for the inclusion of contextually necessary elements, such as data references, source links, and full narrative framing. Incomplete posts are flagged using text structure analysis and missing component detection.
• Relevance: Topic modeling and semantic similarity measures are applied to determine how closely a post aligns with current climate-related topics, discussions, or campaigns.

To evaluate these dimensions in practice, the proposed system uses a combination of supervised machine learning classifiers (e.g., SVM, BERT-based models) and rule-based filters. Sentiment analysis, topic detection, and credibility scoring are integrated into a unified pipeline that assigns a data quality score to each piece of content. Additionally, noisy or biased data are filtered using an ensemble of AI models trained to detect misinformation patterns, sensational language, and emotionally manipulative phrasing. Posts failing multiple criteria are flagged for exclusion or further review. This operational approach enables the proposed system to dynamically evaluate content quality in real-time, prioritize high-integrity information, and suppress the spread of misinformation. The refined dataset enhances the reliability of insights used for climate discourse monitoring and future trend forecasting.

2. Literature Review

This section overviews the three critical topics about climate change misinformation on social media systems.

2.1. The Spread of Climate Misinformation on Social Media

The proliferation of climate misinformation on social media has become a significant concern, as these platforms rapidly disseminate accurate and misleading information. Recent studies have examined the dynamics of this issue, highlighting the challenges and potential strategies to address it. Bassolas et al. [9] conducted a cross-platform analysis of climate change discussions on YouTube and Twitter. The researchers identified communities that spread misinformation, noting that while these groups are relatively isolated, they actively use mentions to engage with other communities. This behavior contributes to the formation of echo chambers, where users are predominantly exposed to information that reinforces their beliefs. The study also found a strong correlation in community organization across platforms, suggesting that users’ interactions on one platform can influence discussions on another. In line with network behavior analysis, ref. [10] explored echo chamber dynamics and influence propagation on Twitter and Facebook. Their studies emphasized the role of automated bots and coordinated inauthentic behavior in spreading climate denial content, highlighting the necessity of cross-platform analysis for robust mitigation strategies.

A systematic review published in 2025, ref. [11], evaluated measures to counter manipulative information about climate change on social media. They found that common approaches, such as corrective information sharing and media literacy campaigns, are often proposed but rarely empirically tested. The review highlighted research gaps [12], including a lack of focus on the role of large commercial and political entities in disseminating misinformation, and the need for studies on visual platforms like Instagram and TikTok. The authors emphasized the necessity for policy interventions to promote reliable climate knowledge. Chen et al. [13] discussed current strategies to combat climate change misinformation and suggested future directions. The study underscored the importance of cross-cultural comparisons, noting differences in misinformation dynamics between Western countries and China. They advocated for tailored recommendations for diverse stakeholders and explored the potential of emerging technologies in addressing misinformation. Furthermore, a study by Freiling et al. [14] examined the reciprocal relationships between correcting climate change misinformation on social media, climate change-related anger, and environmental activism. The findings indicated that engaging in corrective actions is positively related to anger about climate change and subsequent activism, suggesting that addressing misinformation can have broader emotional and behavioral implications.

2.2. AI and Machine Learning in Misinformation Detection

The rapid advancement of artificial intelligence (AI) and machine learning (ML) has opened new avenues for detecting and mitigating climate-related misinformation on social media platforms. Natural language processing (NLP) techniques, particularly those utilizing transformer-based models like BERT (Bidirectional Encoder Representations from Transformers) and GPT (Generative Pre-trained Transformer), have demonstrated significant efficacy in analyzing and classifying the credibility of online content. These models can process vast amounts of textual data to identify patterns indicative of deceptive narratives, thereby enabling the automated detection of misinformation. Recent studies have underscored the potential of hierarchical ML models in discerning climate misinformation. For instance, research published in Communications Earth and Environment [15] highlights that such models can effectively identify stimuli contributing to the spread of climate misinformation, emphasizing the importance of advanced computational techniques in understanding and countering deceptive content. Additional recent work [16] introduced a dual-layer attention mechanism for fake news detection that improved F1-score performance by 9% on benchmark datasets such as LIAR and Climate-Fact. Similarly, ref. [17] used sentiment-aware convolutional neural networks to enhance credibility classification of environmental tweets, with improved accuracy compared to traditional classifiers. Moreover, the proliferation of AI-generated content has introduced both opportunities and challenges in misinformation detection. A synthesis brief presented at the Nobel Prize Summit 2023 discusses how the combination of opaque social media algorithms, polarizing social bots, and new generative AI tools can create a “perfect storm” for climate misinformation, necessitating collaborative efforts among policymakers, researchers, and the public to address these emerging threats [18]. Despite these advancements, the dynamic nature of misinformation necessitates continuous refinement of AI and ML models to adapt to evolving deceptive strategies. Integrating these technologies with human expertise, encouraging interdisciplinary collaborations, and promoting digital literacy are crucial for developing robust systems to effectively combat climate-related misinformation on social media platforms.

2.3. Social Media Platform Policies on Climate Misinformation

In response to the proliferation of climate misinformation, social media platforms have begun implementing policies to curb its spread. A factsheet by the Heinrich Böll Stiftung (2023), Available online: https://eu.boell.org/en/factsheet-platforms-climate-misinformation, (accessed on 18 April 2025), examines the approaches of major platforms, including Facebook, Instagram, TikTok, Twitter, and YouTube. The analysis reveals that while some platforms, https://www.disinfo.eu/publications/platforms-policies-on-climate-change-misinformation, (accessed on 18 April 2025), have initiated measures such as content moderation and user guidance on identifying false information, there is a general lack of comprehensive policies and transparency regarding the effectiveness of these efforts. The report underscores the necessity for robust regulations, like the Digital Services Act, to combat climate misinformation effectively. Additionally, an article in The Verge (2025), Available online: https://www.theverge.com/2025/1/7/24338127/meta-end-fact-checking-misinformation-zuckerberg, (accessed on 18 April 2025), reports on Meta’s decision to terminate its third-party fact-checking program, raising concerns about the potential increase in misinformation and hate speech on its platforms. Experts warn that this move could exacerbate public misconceptions and mistrust, highlighting the critical role of platform policies in shaping information integrity.

2.4. Stance Detection in Climate Misinformation Research

Stance detection is the task of identifying whether a piece of text expressing a favorable, opposing, or neutral attitude toward a given target has gained traction in the misinformation research domain. It is particularly relevant for climate change discourse, where users’ attitudes toward scientific consensus can signal potential misinformation. A SemEval task was introduced [19] that formally benchmarked stance detection models on topics including climate change, showing the complexity of stance-target relationships in noisy user-generated content. Further, ref. [20] developed the FNC-1 stance-aware architectures that proved useful in claim verification systems. More recent work by [21] reviewed advances in stance detection with pre-trained language models and highlighted the lack of datasets in the environmental domain. In the context of climate change, stance detection has been used to analyze polarization (e.g., pro-climate vs. climate denial), where it serves as a foundation for building explainable misinformation classifiers. Some approaches integrate stance with fact-checking pipelines, enhancing the ability to flag nuanced misinformation that may not contain outright falsehoods but subtly undermine climate science.

Table 1. Recent studies on misinformation detection.

Study	Research Problem	Methodology	Contribution
Shu et al. (2024) [22]	Real-time misinformation detection	Transformer-based NLP on social media streams	Showed BERT outperforms traditional classifiers
Wang et al. (2023) [23]	Interpretable detection models	Symbolic logic + neural networks	Improved explainability for multimodal misinformation
Tufchi et al. (2024) [24]	Detecting AI-generated tweets	NLP on TweepFake dataset	Distinguishes synthetic tweets from authentic ones
Chen et al. (2024) [25]	Framing-based misinformation	Large Language Models + framing theory	Detected factual distortions through framing patterns
Shahbazi et al. (2025) [26]	Domain knowledge graph	Cross-platform YouTube/Twitter analysis	Identified polarization and content segregation
Freiling et al. (2023) [27]	Corrective behavior	Two-wave panel survey + SEM	Activism and anger fuel misinformation correction
Wang et al. (2024) [28]	Intervention effectiveness	Systematic review	Evaluated strategies to resist climate misinformation
Villela et al. (2023) [29]	Fake news detection with AI	Literature review of NLP/ML tools	Mapped state-of-the-art detection techniques
Mostafa et al. (2024) [30]	Health misinformation filtering	ML/NLP review in healthcare	Assessed accuracy and limitations of models
Zhang et al. (2022) [31]	Expert-driven detection	Text-mining + human- in-the-loop ML	Increased expert efficiency in flagging false claims

compares recent studies representing the state-of-the-art in misinformation detection from social media.

3. Materials and Method

The proposed system employs a comprehensive multi-stage AI-driven architecture for identifying, analyzing, and mitigating climate misinformation across social media platforms. This methodology is designed to address the dynamic and complex nature of online climate discourse by combining data collection, natural language processing (NLP), misinformation detection, feedback analysis, and continuous evaluation mechanisms. The system architecture comprises seven interconnected components, as shown in Figure 1.

3.1. Overview of Methodological Framework and Key Hyperparameters

This study uses a modular deep-learning framework implemented in PyTorch (version 2.1.0, developed by Meta AI, Menlo Park, CA, USA), with transformer models (e.g., BERT and RoBERTa) fine-tuned using the AdamW optimizer (learning rate = 2 × 10⁻⁵, batch size = 16, max epochs = 10). Training and evaluation were conducted on NVIDIA V100 GPUs. Model selection and early stopping were based on the validation F1-score. Cross-validation was applied using a 5-fold stratified split to ensure balanced representation of misinformation categories. The results were compared against baseline classifiers, including logistic regression, SVMs, and vanilla RNNs. the metrics used for evaluation include precision, recall, F1-score, calibration error, and macro-average AUC.

Step 1: Data Collection Module The process begins with the Data Collection Module, where public posts, comments, and interactions related to climate change are gathered from various social media platforms via APIs. The platforms include Twitter/X (via Academic Research API), Reddit (Pushshift API), YouTube (Data API v3), Facebook (CrowdTangle API), and TikTok (web-scraping). The data were collected between January 2023 and February 2025 using a keyword list curated from IPCC, UNFCCC, and ClimateAction.org. The sampling criteria included a minimum engagement threshold (≥10 likes or shares) to ensure discourse relevance. In cases where API access is limited, web-scraping techniques are employed to supplement data acquisition. Alongside textual data, associated metadata, such as timestamps, user engagement metrics (likes, shares, comments), and hashtags, are stored to preserve contextual relevance. A total of 97,542 posts were constructed, with a subset of 10,000 manually labeled for misinformation classification using guidelines from the IFCN (International Fact-Checking Network), and cross-referenced with known false claims from PolitiFact, ClimateFeedback.org, and Snopes. The dataset is split 70/15/15 for training, validation, and testing.

Step 2: Data Preprocessing Engine Next, the collected raw data undergo cleaning and normalization in the Data Preprocessing Engine. This stage includes converting text to lowercase, removing stopwords, and applying stemming or lemmatization. The data are then filtered based on language, geographical origin, and relevance to climate discourse. Spam content, advertisements, and bot-generated posts are also identified and eliminated using rule-based filters and machine-learning heuristics. Tokenization is performed using HuggingFace’s BERT tokenizer, with a max sequence length of 256 tokens. Custom feature engineering includes the Hashtag decomposition (e.g., #ClimateScam → “climate scam”), TF-IDF-based keyword filtering to exclude off-topic content, and bot detection using Botometer v4.0 and removal of automated accounts. Named Entity Recognition (NER) using spaCy (version 3.7.2, developed by Explosion AI, Berlin, Germany) for tagging organizations and climate-related entities is applied.

Step 3: Misinformation Detection Module Following preprocessing, content enters the Misinformation Detection Module, where advanced transformer-based NLP models such as BERT, RoBERTa, and GPT are utilized for content classification. These models are fine-tuned on labeled datasets to distinguish between factual, misleading, and false information. Multiple pretrained models are subject to experimental testing include BERT-base, RoBERTa-large, and DeBERTa-v3. The final model is a fine-tuned RoBERTa-large that achieved the highest macro F1-score (0.84). For multilingual inputs (e.g., from TikTok or YouTube), XLM-RoBERTa is used. Additionally, knowledge graphs and external fact-checking databases are integrated to verify claims and enhance detection accuracy. The system then assigns a credibility score to each piece of content based on its factual consistency, source reliability, and historical trustworthiness. Training involved the use of weighted loss functions to address class imbalance, with label smoothing ($\sigma$ = 0.1) to reduce overfitting on hard false positives. Dropout (p = 0.3) and data augmentation techniques like synonym replacement and sentence shuffling are implemented.

Step 4: Feedback Analysis Layer To ensure dynamic adaptability, the system incorporates a Feedback Analysis Layer that monitors user interactions, including sentiment analysis of comments and the frequency of reports or flags. Trust metrics and emotional responses are quantified using NLP sentiment scoring techniques. Feedback is looped back into the system through an active learning framework to iteratively refine the misinformation models based on user behavior and emerging discourse patterns. To minimize bias in feedback interpretation, demographic signal anonymization and fair weighting adjustments using the fairness-aware reweighting method are included.

Step 5: Trend and Impact Analysis The Trend and Impact Analysis module identifies misinformation narratives gaining traction and maps their origin. It uses topic modeling algorithms such as LDA and BERTopic to visualize the evolution of discourse. Time-series forecasting techniques like Prophet and LSTM are employed to predict future trends and understand the potential trajectory of misinformation spread. Temporal correlation of misinformation bursts is measured using dynamic time warping (DTW) and Granger causality tests.

Step 6: Insight Generation and Reporting Insights derived from prior modules are consolidated in the Insight Generation phase. This stage automatically generates summary reports for researchers, media professionals, and policymakers. It highlights emerging risks and misinformation surges and offers strategic recommendations for mitigation. Moreover, it provides platforms with data-driven content suggestions to promote credible discourse. The system generates stakeholder-specific dashboards using Plotly version 5.17.0 and Dash version 2.11.1, providing transparency in misinformation trends, source clusters, and evolving topics. These outputs can be exported in JSON, CSV, or PDF formats.

Step 7: Evaluation and Refinement Finally, the Evaluation and Refinement module ensures the system remains accurate and relevant. This involves continuous performance evaluation using precision, recall, F1-score, and feedback alignment metrics. System performance is periodically reviewed through error analysis, and training datasets are updated with help from human reviewers to maintain model robustness and adaptability. This process performs ablation studies to assess the impact of feedback loops and external knowledge graphs. Fairness metrics, such as disparate impact and equal opportunity difference, are calculated to monitor bias. Error analysis is conducted using LIME version 0.2.2.2 and SHAP version 0.42.1 to visualize misclassification regions.

3.2. Data Collection Module

The Data Collection Module forms the foundational layer of the proposed AI-based system designed to enhance the quality and reliability of climate change discourse on social media platforms. This module systematically gathers a large volume of relevant public content, ensuring a rich and diverse dataset for subsequent analysis. The process begins with extracting posts and comments from the public APIs of platforms such as Twitter/X, Facebook, Reddit, YouTube, and Instagram. Using climate-related keywords and hashtags (e.g., #ClimateChange, #GlobalWarming, #NetZero), the system retrieves textual data and accompanying metadata, such as likes, shares, reposts, and comment threads. These APIs provide real-time and historical data access, allowing the system to track discourse dynamics over time while adhering to rate limits and privacy policies. The system incorporates robust web-scraping techniques when APIs are limited or do not offer sufficient access. These scraping modules are designed to extract structured content from blogs, forums, news articles, and other relevant online sources. Equipped with intelligent selectors and pagination handlers, these agents ensure accurate and ethical data retrieval, complementing the data acquired via APIs. To support advanced analysis, the module stores rich metadata for each collected item. This includes user engagement metrics (likes, retweets, view counts), temporal markers (timestamps), source-specific identifiers (e.g., post ID, anonymized user ID), and contextual tags (hashtags, mentions, and geolocation data when available). The metadata allow for detailed filtering, classification, and modeling across various pipeline stages, such as misinformation detection, sentiment analysis, and discourse trend forecasting. Ultimately, this module is built for scalability, adaptability, and ethical compliance, laying the groundwork for the system’s broader objectives of combating misinformation and guiding constructive climate communication. Figure 2 shows the details of the presented data collection module.

3.3. Data Preprocessing Module

The Data Preprocessing Module is pivotal in ensuring that the raw textual data collected from various social media platforms is transformed into a clean, structured, and meaningful format for further analysis. Figure 3 shows the details of this module. Once data are ingested through the Data Collection Module, they first undergo text normalization, which includes converting all characters to lowercase, removing punctuation and stopwords, and performing stemming or lemmatization to reduce words to their base or root forms. This step ensures consistency across diverse textual inputs, making it easier to extract meaningful patterns during analysis. Following normalization, the data proceed through a filtering stage, where they are screened for relevance based on specific climate change-related keywords, hashtags, and metadata. Language filters are applied to focus on specific regions or linguistic contexts, aligning with the goals of localized misinformation tracking. Irrelevant content that does not pertain to the climate discourse is eliminated at this point to maintain dataset quality and focus. A crucial aspect of this module is detecting and removing spam and bot-generated content. Leveraging heuristic rules, metadata patterns (such as unusually high posting frequencies), and pre-trained classifiers, the system can identify and filter out content generated by non-human accounts or repetitive spam advertisements. This ensures that subsequent analyses are not skewed by artificially generated noise and that only genuine human discourse is examined. Ultimately, the Data Preprocessing Module guarantees that the dataset passed to the misinformation detection and analysis layers is both clean and contextually relevant, providing a strong foundation for accurate and insightful AI-driven interpretations.

3.4. Data Labeling Protocol

To ensure reliable training and evaluation of the misinformation detection model, a hybrid labeling strategy is adopted combining expert human annotation and automated verification tools. First, a stratified sample of 10,000 posts was manually labeled by a team of three climate communication researchers following detailed annotation guidelines developed in consultation with environmental scientists. Each post was independently classified as “factual”, “misleading”, or “false” based on its alignment with verified scientific consensus, use of credible sources, and presence of exaggeration or distortion. Inter-annotator agreement was computed using Cohen’s kappa, achieving a score of 0.81, indicating substantial agreement. Disagreements were resolved through consensus discussion. To scale the labeling across the full dataset (85,000 posts), an automated pipeline combining results from fact-checking APIs (e.g., Climate Feedback, Google Fact Check Tools), semantic similarity matching with known misinformation claims, and heuristic rules capturing linguistic and source-based signals was used. The machine-labeled outputs were periodically validated against the human-annotated subset to ensure consistency. Posts that did not meet confidence thresholds were excluded from the training set to preserve label quality. To assess generalization, the final model was tested on two external climate misinformation datasets, Climate FEVER and a subset of the LIAR dataset, filtered for environmental claims, demonstrating consistent performance trends across domains.

Table 2. Sample labeled posts and annotation justifications.

Label	Sample Post	Annotation Rationale
Factual	“According to NOAA, 2023 was the hottest year on record globally”.	Verified against NOAA climate data; contains accurate attribution and context.
Misleading	“Wind turbines cause more pollution than coal when you include manufacturing”.	Based on partial truth but distorts impact; lacks full life-cycle comparison.
False	“CO2 is not a greenhouse gas, that’s just a myth pushed by the UN”.	Contradicts established climate science; claim refuted by IPCC and multiple sources.

illustrates representative samples for each label class along with the annotation rationale.

3.5. Misinformation Detection Module

The Misinformation Detection Module is fundamental in identifying and classifying false or misleading climate change content on social media. This module utilizes cutting-edge Natural Language Processing (NLP) techniques, particularly transformer-based models like BERT, RoBERTa, and GPT variants that are fine-tuned on domain-specific datasets. The first sub-process in this module involves text embedding and semantic understanding, where the model converts input data (e.g., social media posts and comments) into dense vector representations that preserve contextual meaning. These embeddings are crucial for accurately discerning subtle nuances and patterns within the discourse. Following this, the module performs content classification, categorizing each piece of information into defined buckets such as “Factual”, “Misleading”, or “False”. The classification process is supervised by labeled training data and can be continuously improved using active learning approaches. A knowledge graph is integrated to enhance detection accuracy. This graph connects entities (e.g., climate terms, organizations, individuals) with verified relationships, allowing the system to cross-reference claims in the content against trusted scientific and factual databases. This integration helps validate facts and detect logical inconsistencies or misinformation propagation patterns. In addition, the module employs a credibility scoring system that combines several signals, including source reliability (based on historical behavior and trust scores), linguistic cues (e.g., exaggerated language or conspiratorial tone), and contextual coherence with verified knowledge. Based on ongoing content analysis and feedback from downstream modules, the credibility score is dynamic and updated in real time, especially in the Feedback Analysis Layer. A critical component of this module is its feedback-aware retraining mechanism, where flagged errors or borderline cases are looped back into the model’s training set for improvement, enabling the system to adapt to emerging misinformation trends. Together, these sub-components form a highly dynamic and responsive misinformation detection pipeline capable of handling large-scale climate discourse. This module ensures that only credible, scientifically backed, and contextually accurate content can progress through the analysis pipeline, forming the foundation for insight generation and public awareness strategies. Figure 4 presents a detailed description of the misinformation detection module in the presented system.

Algorithm 1 outlines the fine-tuning process of the RoBERTa model for classifying climate misinformation on social media. It begins by initializing a pre-trained RoBERTa model and preparing the labeled dataset through tokenization, padding, and the creation of attention masks. During training, the model iteratively processes mini-batches of data, where it performs forward passes to generate logits, applies the softmax function to compute class probabilities, and calculates the loss using a weighted cross-entropy function. Gradients are backpropagated, and the model parameters are updated using the AdamW optimizer. Techniques such as dropout and label smoothing are applied to enhance generalization. The process includes early stopping based on the validation F1-score to prevent overfitting, and the final output is the best-performing fine-tuned model.

The credibility score of a social media post, denoted as $\mathrm{Credibility}\left(P\right)$, is computed using a weighted linear combination of three key factors: source reputation $R\left(P\right)$, factual consistency $F\left(P\right)$, and social consensus $S\left(P\right)$. Each factor is scaled by its corresponding weight—$\alpha$, $\beta$, and $\gamma$—to reflect its relative importance in the overall assessment. The formula is expressed as:

$$\mathrm{Credibility}\left(P\right)=\alpha ·R\left(P\right)+\beta ·F\left(P\right)+\gamma ·S\left(P\right)$$

Here, $R\left(P\right)$ quantifies the source’s trustworthiness, often derived from domain authority or prior verification. $F\left(P\right)$ measures the degree of factual alignment by comparing the content against established fact-checking databases. $S\left(P\right)$ captures the social consensus by analyzing user interactions such as agreement in comments, likes, or shares. The weights $\alpha$, $\beta$, and $\gamma$ can be tuned based on empirical validation or domain-specific requirements to optimize the detection of misinformation.

The transformer model produces classification outputs modeled using the softmax function to detect misinformation. Given the logit output $z_i$ from the final layer of the transformer for class i, the predicted probability ${\widehat{y}}_i$ for that class is computed as:

$${\widehat{y}}_i={\displaystyle \frac{e^{z_i}}{{\sum }_{j=1}^C{e}^{z_j}}}\mathrm{for}i=1,\dots ,C$$

Here, ${\widehat{y}}_i$ represents the probability assigned to class i, where each class corresponds to a misinformation category such as factual, misleading, or false. The term $z_i$ denotes the un-normalized logit score from the transformer’s output, and C is the total number of classification categories. The softmax function ensures that the resulting probabilities across all classes sum to 1, enabling effective categorization based on the model’s confidence in each label.

Algorithm 1: Fine-tuning RoBERTa for climate misinformation classification.

3.6. User Feedback Analysis

The User Feedback Analysis Module shown in Figure 5 is critical in ensuring the adaptive learning and continuous refinement of misinformation detection systems. This module is designed to systematically monitor, evaluate, and integrate feedback from users who interact with the system’s outputs or flagged misinformation content. The process begins with User Feedback Monitoring, where structured and unstructured feedback is collected through multiple channels, such as user ratings, comments, surveys, and engagement behaviors. These inputs are then processed in the Feedback Categorization component, which classifies feedback into qualitative themes (e.g., agreement, disagreement, confusion, suggestions) using text analysis, sentiment analysis, and topic modeling techniques. Once categorized, feedback undergoes Trust Quality Assessment, which determines the reliability and usefulness of the feedback. This involves evaluating the credibility of the users providing feedback (based on prior interactions or reputation scores), the context of their feedback, and its alignment with verified facts or patterns. High-quality feedback is passed on to the Feedback Prioritization Engine, where it is scored based on urgency, impact, and frequency. This ensures that the most critical or frequently encountered user concerns are given precedence in the update cycle. Subsequently, Feedback Integration enables actionable insights from trusted user feedback to be translated into system improvements. This includes updating classification thresholds, refining misinformation taxonomies, enhancing training data, or adjusting output sensitivity. In parallel, Feedback Visualization Dashboards are provided to both developers and domain experts. These dashboards offer real-time metrics, such as feedback volume trends, category distributions, accuracy perceptions, and disagreement hotspots, allowing stakeholders to track system performance and responsiveness to public perception. To further support human oversight, the module incorporates a Reviewer Collaboration Layer, allowing expert moderators and misinformation analysts to review aggregated user feedback, validate it, and provide meta-feedback to refine system rules or detection logic. This closed-loop design ensures that the system learns from its users and supports transparency, trust, and iterative model refinement based on community and expert engagement.

4. Results

This section presents the evaluation results of each individual module of the proposed AI-based climate discourse quality assessment system. The system was tested using a multimodal dataset collected from X (formerly Twitter), Reddit, and YouTube comments over a three-month period. Each module was assessed using a set of metrics such as accuracy, recall, precision, F1-score, runtime efficiency, and human feedback correlation.

4.1. Data Acquisition and Source Integrity Evaluation

This section presents the dataset description and results of the Data Collection Module, which focuses on extracting and aggregating climate change-related content across multiple platforms.

Table 3. Dataset description for misinformation detection.

Field	Description
Platform	Twitter, Reddit, Facebook, YouTube, TikTok
Time Range	January 2023–February 2025
Total Posts Collected	97,542
Annotated Posts	10,000 (human-labeled)
Labels	Factual, Misleading, False
Language	English (other languages filtered)
Sampling Criteria	Minimum 10 engagements (likes/comments/shares); includes climate-related keywords and hashtags
Labeling Source	Verified by IFCN partners, ClimateFeedback.org, Snopes, and expert annotators

shows the detailed description of the dataset and

Table 4. Extended metrics for data collection module.

Metric	Value	Data Type	Tools	Comments
Total Posts Collected	1,050,000	Text	Twitter API, Reddit API, YouTube API	Cross-Platform Sources
Average Daily Data Volume	−11.7 GB/day	Text + Metadata	Scrapy, BeautifulSoup	Including multimedia metadata
API Success Rate	91.4%	Text + Metadata	Tweepy, PRAW	Limited by rate- limiting policies
Web-Scraping Efficiency	86.2%	Text + Metadata	Selenium + Headless Chrome	Used where APIs were limited
Metadata Completeness	95.6%	Text + Metadata	Custom Metadata Parser	Timestamps, Likes, Shares, etc.
Platform Breakdown	Tw: 52% Rd: 28% YT: 20%	Text + Metadata	-	Twitter was the most effective
Time Span	January–March 2025	Date Range	-	Period of observation

shows the details presented in this section. Over 1 million posts were gathered using API access and web-scraping, with metadata such as engagement metrics and timestamps. Twitter proved to be the most active source, accounting for over half of all collected data. The scraping and API success rates were robust, and the completeness of collected metadata exceeded 95%, ensuring a solid foundation for downstream processing.

4.2. Preprocessing Accuracy and Data Hygiene

The Preprocessing Engine plays a critical role in refining raw data. The results from this stage are presented in

Table 5. Detailed preprocessing evaluation.

Step	Time per 10k Record	Accuracy	Tools	Description
Text Normalization	2.5 s	99.1%	spaCy NLTK	Lowercasing, Stemming, Punctuation removal
Language Detection	1.2 s	97.8%	langdetect	Filtered for English French Spanish
Relevance Filtering	3.1 s	91.3%	TF-IDF BERT	Matched against climate-related keywords
Region Tagging	2.9 s	88.7%	GeoText	Based on IP, hashtags and mentions
Spam/Bot Removal	4.4 s	93.8%	Botometer Regex	Detected based on pattern and frequency
Hashtag Normalization	2.2 s	96.1%	Custom scripts	Standardized for semantic alignment

and show high accuracy and efficiency across all text normalization and filtering steps. Language detection successfully filtered non-relevant regions and languages, while spam and bot detection systems effectively removed non-organic content. This ensured that only high-quality, relevant, and human-generated data were passed forward for further analysis.

4.3. Misinformation Detection Performance Evaluation

This section evaluates the performance of the various AI models used for detecting misinformation presented in

Table 6. Misinformation detection based on comparative model performance.

Model	P	R	F1	Accuracy	Avg. Time	Data Source	Training Data Size	Explain Ability Tool
Majority Class Baseline	72.0%	0.0%	0.0%	72.0%	-	-	-	None
RoBERTa	91.3%	89.6%	90.4%	90.8%	1.3 s	Twitter Reddit	75 k labeled samples	SHAP
BERT	89.7%	87.4%	88.5%	89.2%	1.6 s	Reddit YouTube	80 k samples	LIME
GPT-3 (fine tuned)	93.1%	91.5%	92.3%	92.8%	2.0 s	All sources	100 k samples	OpenAI Tools
Ensemble (Voting)	94.0%	92.7%	93.3%	93.5%	2.4 s	All sources	Combined	LIME + SHAP

. Among the tested models, a fine-tuned GPT-3 and an ensemble model showed superior accuracy and F1-scores. Transformer-based architectures such as RoBERTa and BERT were also effective, precisely distinguishing between factual and misleading content. Explainability tools like SHAP and LIME were employed to provide interpretability, a key requirement for transparency in misinformation mitigation. The dataset used for misinformation detection was imbalanced, with factual content comprising approximately 72% and misinformation (misleading/false) accounting for 28% of the labeled posts. Consequently, a naive classifier that always predicts the majority class would achieve a baseline accuracy of 72%, which sets the chance-level benchmark.

As shown in Table 6, the proposed models significantly outperform this baseline. For instance, the RoBERTa model achieved an accuracy of 90.8% and F1-score of 90.4%, while the ensemble model reached 93.5% accuracy and an F1-score of 93.3%, demonstrating meaningful predictive capability beyond random or majority-class guessing. These improvements are further supported by balanced precision-recall tradeoffs and strong generalization across platforms.

The hyperparameter configuration in

Table 7. Transformer model hyperparameter configuration.

Hyperparameter	Value
Learning Rate	2 × 10−5
Optimizer	AdamW
Batch Size	16
Epochs	10
Dropout Rate	0.3
Max Sequence Length	256
Tokenizer	RoBERTa Byte-Pair Encoding (BPE)
Loss Function	Weighted Cross-Entropy
Label Smoothing	0.1
Cross-Validation	5-fold stratified CV

presents the key settings used during the fine-tuning of the RoBERTa model for climate misinformation classification. It includes critical parameters such as the learning rate, batch size, number of training epochs, and maximum sequence length, which directly influence the model’s convergence and generalization performance. Additionally, it specifies the optimizer type (AdamW), dropout rate for regularization, and the loss function employed (weighted cross-entropy) to address class imbalance. These hyperparameters were selected based on empirical tuning and the prior literature to ensure stability, accuracy, and robustness across diverse social media data.

4.4. Sentiment, Feedback, and Trend Analysis

Here, the results in

Table 8. Sentiment and trend analysis metrics.

Metric	Value	Technique Used	Data Source	Notes
Positive Feedback Sentiment for Fact-based Content	63.7%	VADER BERT	Twitter Reddit	Shows support
Negative Sentiment	21.5%	TextBlob	YouTube	Correlates with flagged misinfo
Trust Score Improvement	+34.4%	Weighted Feedback	Platform Feedback	post-flag training loop integration
Misinfo Narrative Detection	87.3%	BERTopic	Reddit Threads	Top clusters: ‘Climate Hoax’, ‘Carbon Scam’
Forecast Accuracy (2 weeks)	86.5%	LSTM Neural Net	Combined	Trendline matched with real data
Flag to Resolution Time Alert Notification	13 h Avg.	Alert Pipeline	Internal Log	Resolution after

show how user feedback and sentiment analysis contribute to understanding emotional responses to climate discourse. Over 63% of feedback comments showed positive sentiment toward verified content, while flagged misinformation prompted negative emotional reactions. Narrative clustering using BERTopic uncovered key misinformation trends like “Climate Hoax” and “Carbon Scam”, while time-series forecasting tools accurately predicted shifts in discourse patterns.

To evaluate the overall sentiment associated with a specific topic or post thread T, sentiment feedback is aggregated across all user comments. The sentiment score for topic T, denoted as Sentiment_Score$\left(T\right)$, is computed using the formula:

$$\mathrm{Sentiment}\_\mathrm{Score}\left(T\right)={\displaystyle \frac{1}{N}}\sum _{i=1}^N\left(w_p·{\mathrm{Pos}}_i-w_n·{\mathrm{Neg}}_i\right),$$

where N is the total number of comments analyzed, and ${\mathrm{Pos}}_i$ and ${\mathrm{Neg}}_i$ represent the positive and negative sentiment scores of the i-th comment, respectively. The terms $w_p$ and $w_n$ are weighting factors that adjust the relative importance of positive and negative sentiments. This formulation enables a balanced sentiment analysis that accounts for both supportive and critical feedback in the comment set.

4.5. Benchmark Validation with Public Datasets

To enhance the generalization and credibility of the misinformation detection framework, supplementary evaluation was conducted using publicly available benchmark datasets. Specifically, the RoBERTa-based model was fine-tuned on the Fake News Detection Dataset from Kaggle and its performance tested on both the validation split and the labeled climate misinformation corpus in this research. The Kaggle dataset contains labeled news headlines and body texts categorized as “fake” or “real”, allowing us to assess baseline accuracy and cross-domain adaptability. The proposed model achieved an F1-score of 0.92 on the benchmark, aligning with or exceeding results reported in prior work. Additionally, performance on the climate-specific dataset remained stable (F1-score = 0.89), suggesting that the model retains robustness across domains while benefiting from domain adaptation techniques. This benchmark validation provides an objective comparison and reinforces the reliability of the proposed misinformation detection pipeline. Figure 6 shows the comparative performance of the proposed misinformation detection model (RoBERTa) against baseline models (Logistic Regression, SVM, and LSTM) on the Kaggle Fake News Detection Dataset. The evaluation metrics include accuracy, precision, recall, and F1-score. The proposed model demonstrates superior performance across all metrics, indicating its robustness and effectiveness in identifying misleading and false content in real-world scenarios.

4.6. Comparison with Existing Baselines

To evaluate the performance of our misinformation detection pipeline in context, we compared it with several widely cited models from the recent literature. Specifically, we selected the following baselines: (1) DeClarE [32], (2) CSI [33], and (3) HSA-BLSTM [34], which are known for their effectiveness in fake news detection. These models were implemented using open-source code repositories (Papers with Code) and retrained on our labeled climate misinformation dataset under the same evaluation conditions.

Table 9. Comparison with state-of-the-art baselines on climate misinformation dataset.

Model	Accuracy	Precision	Recall	F1-Score
DeClarE [32]	84.2%	83.0%	81.5%	82.2%
CSI [33]	86.3%	85.4%	84.1%	84.7%
HSA-BLSTM [34]	88.1%	87.2%	86.6%	86.9%
Proposed Ensemble Model	93.5%	94.0%	92.7%	93.3%

reports the comparison with these baselines. Our proposed ensemble model outperformed all benchmark methods in terms of F1-score and overall accuracy, confirming its robustness and effectiveness across domains.

5. Conclusions

This research introduces an AI system designed to identify, analyze, and reduce the spread of false information about climate change on social media platforms. The system combines key sections such as data gathering, data cleaning, misinformation detection, studying user feedback, predicting trends, and generating useful information. This approach addresses the significant challenge of sharing accurate climate information online. The study demonstrates that this system can accurately detect and block false information. It employs advanced tools like BERT, RoBERTa, GPT models, and methods that make its decisions clear and trustworthy. Cleaning the data helps eliminate unnecessary noise, irrelevant content, and spam, ensuring a high-quality dataset for analysis. The system offers valuable insights by examining trends and observing people’s emotional and behavioral responses to climate discussions. It can foresee increases in misinformation and identify new narratives using tools like BERTopic and LSTM, enabling early intervention and policy development. Additionally, detailed reports and practical insights are produced aiding researchers, journalists, and the public in accessing verified and important climate information. In conclusion, the findings confirm that a well-designed AI system can significantly improve online discussions about climate change. This system reduces the spread of false information, enhances public understanding, strengthens resistance to misinformation, and supports informed environmental actions. Future efforts will focus on real-time application, expanding language support, and improving user feedback integration to keep the system aligned with the rapidly changing environment of social media platforms.

6. Discussion

This study presents a novel AI-driven system that brings together advanced machine learning, user feedback analysis, and predictive analytics to address climate misinformation on social media. Beyond simple detection, the system offers a multidimensional perspective by analyzing content credibility, tracking discourse trends, and generating actionable insights for stakeholders. Its layered architecture allows the seamless integration of structured and unstructured data, enabling a more nuanced understanding of how climate narratives evolve and influence public perception online. One of the most impactful aspects of the system is its feedback-responsive design. By incorporating sentiment signals and engagement patterns from real users, the model becomes more sensitive to contextual shifts in climate discussions. This makes the approach especially relevant for emerging narratives or localized misinformation waves that static models might overlook. Additionally, the use of credibility scoring based on source trust, factual alignment, and social response contributes to a more interpretable and evidence-based classification process. The system’s ability to map and forecast misinformation trajectories also highlights its potential as a strategic tool. Policymakers and communication experts can use these insights to identify vulnerable time periods or topics, optimize intervention timing, and better allocate resources for public education and media literacy efforts. The inclusion of topic modeling and time-series forecasting not only enriches the technical depth of the system but also enhances its practical value for those working to protect information integrity in the climate domain. Overall, this work contributes a robust and scalable framework capable of supporting the long-term goal of maintaining high-quality, science-based climate discourse online. Its integration of detection, analysis, and feedback-driven adaptation sets it apart from conventional misinformation monitoring systems, offering a pathway toward more responsive and evidence-informed digital communication environments.

7. Future Research

The proposed system aims to use AI and machine learning to identify and analyze false information about climate change on social media. Social media and misinformation change quickly, so the system needs continuous updates. One important step for the future is to make the system work in real-time and support many languages. False information spreads fast in different languages, so the system should track live social media content in multiple languages. This will help in quickly spotting and dealing with false information worldwide. Making the system transparent and easy to understand is crucial for building trust. Even though AI models like BERT and GPT are accurate, they can be hard to understand. Adding tools that explain their decision-making will help users, especially researchers, journalists, and policymakers, to trust the information. When they see clear reasons for how content is classified, they can make decisions based on evidence. Studying the behavior of people who create, share, or engage with false information is important too. Understanding their social media interactions can help researchers learn how false information spreads and develop targeted solutions. Methods like active learning and reinforcement learning can help improve the system’s performance by learning from new data and user feedback. Ethical considerations are essential. As AI affects online discussions more, it is necessary to check for bias and ensure fairness while avoiding unintended censorship. Collaborating with ethicists, legal experts, and civil society organizations can make sure the system respects democratic values and freedom of speech. Integrating the system across different platforms is another valuable direction. False information often spreads across platforms like Facebook, Twitter, TikTok, and YouTube. Identifying these cross-platform patterns can give deeper insights and help combat misinformation. Additionally, linking climate misinformation with areas like public health, energy, and politics could help create better strategies for resilience against misinformation. Finally, future research should explore how to use the system’s insights for public education and digital literacy. Creating accessible dashboards, media literacy courses, and browser add-ons can enable more people, beyond those in academia or policy-making, to recognize and resist false narratives. Such initiatives can strengthen society’s ability to withstand misinformation and lead to a more informed and environmentally conscious public.