GenAI-Powered Framework for Reliable Sentiment Labeling in Drug Safety Monitoring

Abstract

The analysis of medical data presents an opportunity for healthcare systems to support decision-making and improve patient outcomes. In this context, the automated analysis of user-generated drug reviews offers a promising approach for monitoring medication safety, understanding patient experiences, and detecting potential adverse effects in real time. This study advances sentiment analyses for pharmacovigilance by introducing a data-centric framework that incorporates a GenAI-powered labeling system for reliable and interpretable data annotation. A corpus of 213,869 user-generated drug reviews was processed through a hybrid labeling pipeline that reconciles user ratings, lexicon-based polarity, zero-shot transformer predictions, and GPT-5.2 as a fallback mechanism. This strategy enables the resolution of sentiment ambiguity, particularly the frequent misalignment between user-assigned ratings and underlying textual sentiment, by leveraging contextual understanding rather than relying solely on numerical scores. Drug review representations are enhanced using the Qwen3-Embedding-0.6B model, allowing improved capture of semantic nuances. Evaluated through 10-fold stratified cross-validation, the proposed labeling framework combined with a Random Forest classifier achieves a classification accuracy of 96.45%, with per-class analysis confirming consistent performance across all sentiment categories. Cross-source validation on an independent drug review dataset of 4091 reviews and a threshold sensitivity analysis further support the robustness and generalizability of the proposed approach.

1. Introduction

In today’s healthcare landscape, drug safety has become a cornerstone of public health. Although clinical trials are designed to thoroughly evaluate the safety and effectiveness of new pharmaceuticals, they are constrained by limited time frames, relatively small participant groups, and restricted demographic representation. Consequently, once medications reach the general population, previously undetected side effects and adverse reactions often surface, highlighting the need for strong post-marketing surveillance systems [1]. However, traditional pharmacovigilance methods frequently struggle with issues such as underreporting, reporting biases, and slow feedback processes, which reduce their ability to effectively monitor real-world drug safety.

In this context, advancements in artificial intelligence (AI), particularly generative AI and natural language processing (NLP), offer opportunities to improve drug safety monitoring. By analyzing user-generated content such as drug reviews, social media discussions, and online health forums, it becomes possible to extract timely insights into patient experiences. These reviews frequently capture first-hand reports of side effects, drug efficacy, and overall satisfaction, making them a valuable source for post-marketing safety signals. Integrating patient narratives into pharmacovigilance workflows not only enhances clinical awareness but also promotes patient-centric decision-making by highlighting real-world experiences that often remain invisible to healthcare providers [2].

Sentiment analysis methodologies have shown substantial promise in this domain. Historically reliant on lexicons and rule-based systems, sentiment analysis has evolved to incorporate deep learning models capable of capturing linguistic nuance and context. Recent advancements in transformer-based architectures [3] have significantly improved performance in general-purpose sentiment classification tasks. However, applying such methods to healthcare data, particularly drug reviews, presents unique challenges. One prominent issue is the inconsistency between user-assigned numerical ratings and the actual sentiment conveyed in the text. Reviews may contain conflicting cues, such as high ratings paired with descriptions of adverse effects, thereby complicating the labeling process and model training. Consequently, traditional approaches that rely solely on ratings are prone to introducing noise, leading to a decline in classification accuracy.

Addressing these limitations requires more nuanced sentiment labeling frameworks, ones that combine model predictions with user signals and incorporate mechanisms to resolve ambiguity. This paper proposes a hybrid labeling system designed specifically for sentiment analysis in the context of drug reviews. By leveraging a combination of rule-based logic, multiple sentiment classifiers, and zero-shot classification using large language models (LLMs) such as GPT-5.2 as a judge, the framework seeks to generate consistent and explainable sentiment labels. This approach may support healthcare systems in detecting and responding to safety concerns more effectively.

The problem addressed in this study centers on the challenge of generating accurate and explainable sentiment labels from user drug reviews, where ambiguity in language and inconsistency between ratings and narrative content often hinder traditional machine learning pipelines. Existing sentiment classifiers fail to resolve this divergence effectively, which undermines the quality of labeled datasets used for pharmacovigilance and downstream analytics.

The structure of this paper is organized as follows. Section 2 reviews related work on sentiment analysis in healthcare and drug safety applications. Section 3 presents the materials and methods used, including the dataset description, preprocessing and semantic representation procedures, the labeling algorithm, and the overall system architecture. Section 4 reports the experimental results. Section 5 discusses the findings and their implications. Finally, Section 6 concludes this paper by summarizing its main contributions and outlining future research directions.

2. Related Work

The utilization of sentiment analysis across various domains has seen significant advancement, yet its application within the realm of pharmaceutical reviews is relatively nascent. While several sectors, including business and hospitality, have integrated sentiment analysis to enhance customer insights and service delivery, the pharmaceutical sector lags behind, particularly in the nuanced area of drug reviews. Drug review sentiment analysis is especially challenging due to domain-specific vocabulary, conflicting sentiment patterns, and inconsistencies between user ratings and textual narratives. A few researchers have begun to investigate this potential, applying advanced analytical methods to enhance understanding and insights derived from drug-related feedback.

In 2019, Alam and Yao [4] highlighted the importance of preprocessing pipelines for sentiment analysis. Their study showed that cleaning and normalization significantly influence model accuracy, with a CNN–LSTM ensemble achieving 86.7%. This emphasizes that robust preprocessing remains a foundational step for healthcare text mining. By 2020, research turned toward neural architectures capable of modeling contextual semantics. Han et al. [5] proposed a transfer learning-based multi-task sentiment framework using a dual Bi-GRU with attention, obtaining 78.26% accuracy on the DrugLib.com dataset. In parallel, Vijayaraghavan and Basu [6] investigated supervised ML and DL approaches on Drugs.com, where their artificial neural network achieved 93.85% accuracy, underlining the potential of hybrid ML–DL strategies. In the same year, Colón-Ruiz and Segura-Bedmar [7] compared CNNs, LSTMs, and hybrid models, demonstrating that combining BERT [8] with Bi-LSTM led to significant improvements, achieving 90.46% accuracy in three-class sentiment classification.

Building on these results, post-2020 research expanded in two directions: combining traditional machine learning features with deep learning techniques and leveraging transformer architectures for domain-specific review analysis. Garg [9] introduced a sentiment-driven recommendation system in 2021 using TF-IDF with LinearSVC, achieving 93% accuracy. In 2022, Uddin et al. [10] tested several classifiers on UCI drug reviews, with Random Forest achieving the highest binary accuracy at 94%. In 2023, Lee et al. [11] proposed a hybrid machine learning–deep learning framework that combined lexicon-based annotation (TextBlob) with CNN–LSTM and classical ML models. Their ensemble CNN–LSTM achieved 96% accuracy on a large Drugs.com dataset while explicitly addressing rating–text contradictions and validating performance with statistical tests.

The summary of previous works presented in Table 1 indicates that, over the past few years, sentiment analysis of drug reviews has undergone a clear methodological progression, transitioning from preprocessing-focused methods to hybrid architectures, transformer-based approaches, and, more recently, LLM-assisted solutions.

Table 1. Summary of related work.

Ref.	Year	Dataset	Methods/Models	Accuracy
[4]	2019	Askpatient.com	Preprocessing pipelines + CNN–LSTM ensemble	86.7%
[5]	2020	DrugLib.com	Double Bi-GRU + Attention	78.26%
[6]	2020	Drugs.com	ANN + ML (LR, RF, etc.)	93.85%
[7]	2020	Drugs.com	CNN, LSTM, BERT + Bi-LSTM	90.46%
[9]	2021	Drugs.com	TF-IDF + Linear SVC	93%
[10]	2022	UCI drug reviews	ML models (RF, SVM, etc.)	94%
[11]	2023	Drugs.com	Hybrid ML–DL (TextBlob, CNN–LSTM)	96%

Taken together, these studies outline a clear trajectory: from preprocessing and CNN–LSTM ensembles [4] to attention-based architectures [5] and ANN baselines [6], to hybrid transformer models [7], classical-feature hybrids [9,10], and integrated frameworks combining lexicon-based sentiment tools with neural architectures [11], and, finally, to LLM-assisted pipelines. While this progression demonstrates substantial methodological advancement, a critical limitation persists across the body of work. The majority of contributions have concentrated on model architecture, feature engineering, or transfer learning, yet comparatively little attention has been devoted to the quality and reliability of the labels that underpin training and evaluation. Specifically, Vijayaraghavan and Basu [6] and Uddin et al. [10] relied entirely on user-assigned ratings as ground truth without examining whether these ratings accurately captured textual sentiment. Similarly, Lee et al. [11] adopted TextBlob polarity as the labeling source, inheriting the well-documented limitations of lexicon-based methods in domain-specific and negation-rich medical language. These labeling strategies assume a consistent mapping from ratings or lexicon scores to sentiment, an assumption that is frequently violated in drug reviews where patients may assign high ratings despite reporting severe side effects.

More recently, the rapid development of LLMs has opened new avenues for data annotation and sentiment analysis in healthcare. Zhang et al. [12] demonstrated that LLMs can perform zero-shot and few-shot sentiment classification with competitive accuracy when provided with appropriately designed prompts, while Gilardi et al. [13] showed that ChatGPT (GPT-3.5-turbo)-generated annotations can match or exceed crowd-worker quality across several text classification tasks. In the biomedical domain, Hu et al. [14] applied prompt-engineered LLMs to clinical named entity recognition, reporting promising results but also highlighting the need for human validation and domain-specific prompt design. These findings suggest that LLMs offer a scalable alternative to manual annotation, yet their outputs still require systematic quality control, particularly in safety-critical applications such as pharmacovigilance.

A parallel line of research has addressed label quality through the lens of programmatic weak supervision and data-centric AI. The Snorkel framework [15] introduced the concept of labeling functions, heuristic rules that generate noisy labels, which are subsequently aggregated via a generative model. Zhang et al. [16] extended this paradigm with WRENCH, a benchmark for evaluating weak supervision pipelines across diverse NLP tasks. More broadly, the data-centric AI movement [17] has argued that systematic improvements to data quality, particularly label quality, often yield larger gains than architectural innovation alone. However, the application of these principles to pharmaceutical sentiment analysis remains limited, with most health NLP studies continuing to use uncurated labels derived from crowd ratings or single-source lexicon tools.

Moreover, although prior work on dataset reusability [18] highlighted the challenges of cross-domain portability, far less consideration has been given to the equally critical issue of label reliability within single-domain corpora, a gap that this study directly addresses. Since mislabeled ground truth can propagate errors regardless of model sophistication, improving label reliability is a prerequisite for advancing sentiment analysis in the pharmaceutical domain. The approach proposed in this study operates at a fundamentally different stage from prediction-level ensemble methods such as stacking or boosting. Rather than combining classifier outputs at inference time, it reconciles multiple heterogeneous annotation signals, user ratings, lexicon-based polarity, zero-shot transformer predictions, and LLM-based judgments before training to produce higher-fidelity labels. This positions the proposed framework between traditional ensemble labeling and programmatic weak supervision [15]. It adopts Snorkel’s philosophy of aggregating multiple noisy sources while replacing the generative noise model with an explicit, interpretable decision cascade that records full provenance for each label.

3. Materials and Methods

3.1. Dataset Description and Visualization

The primary dataset used in this study originates from the publicly available Drug Review Dataset, provided through the KUC Hackathon Winter 2018 and hosted on Kaggle [19,20]. It contains user-generated reviews of various drugs, along with associated metadata. In total, the dataset contains 215,063 instances. After removing duplicate entries and records with missing values during preprocessing, the final corpus comprised 213,869 reviews, providing a comprehensive basis for sentiment analysis of drug reviews in healthcare applications.

As shown in Table 2, the dataset consists of six primary attributes:

Table 2. Dataset visualization.

DrugName	Condition	Review	Rating	Date	UsefulCount
Levonorgestrel	Emergency Contraception	He pulled out, but he chummed a bit in me. I took the pill within 12 h. Now it’s been 10 days, no period, just cramps. I am really scared.	1	7-Mar-17	5
Pentasa	Crohn’s Disease	I had Crohn’s with a resection 30 years ago. I was taking Pentasa for a long time, but it stopped working for me.	4	6-Jul-13	8
Ativan	Panic Disorder	Honestly, I have been taking Ativan for 2 years. It helps, but it also makes me sleepy and forgetful sometimes.	6	1-Jun-15	47
Valsartan	Left Ventricular Dysfunction	It has no side effect, I take it in combination of 9 Bystolic 5 Mg and Fish Oil.	9	20-May-12	27
Aripiprazole	Bipolar Disorder	Abilify changed my life. There is hope. I was able to go back to school and finish my degree. It gave me my life back.	10	14-Mar-15	32

• DrugName: The name of the prescribed drug;
• Condition: The associated medical condition;
• Review: The free-text user review;
• Rating: A satisfaction score from 1 to 10;
• Date: The date when the review was posted;
• UsefulCount: The number of users who found the review useful.

3.2. Dataset Preprocessing and Semantic Representation

The preprocessing pipeline was designed to clean and normalize noisy user-generated drug reviews to support reliable sentiment label generation and downstream analysis [4]. Raw reviews were first passed through a custom text-cleaning function that removed irrelevant strings, HTML tags, and entities while also handling redundant whitespace, line breaks, and bracketed characters. This step ensured that all reviews were standardized into plain, continuous text. Common English stop words were subsequently removed to further reduce linguistic noise for lexicon-based labeling methods, focusing the analysis on semantically meaningful words rather than high-frequency function terms. Duplicate entries and records with missing values were also excluded, yielding a consistent and compact dataset.

Next, sentiment labels were generated using multiple complementary approaches. User ratings (1–10) were normalized into three categories, Negative (1–4), Neutral (5–7), and Positive (8–10), to provide rating-based sentiment labels. In addition, the lexicon-based method (TextBlob) [21] was used to compute polarity scores and assign sentiment categories based on fixed thresholds: reviews with a TextBlob polarity score below $-0.05$ were labeled as Negative, those above $+0.05$ were labeled as Positive, and the remainder was Neutral. Additionally, two different LLM-based zero-shot sentiment classifiers, BART and GPT-5.2, were used to enrich each review with multiple sentiment perspectives. The BART classifier (Facebook/bart-large-mnli) [22,23] also provided a confidence score, which was leveraged in the proposed labeling framework. The resulting dataset therefore included cleaned text, mapped rating categories, lexicon-derived sentiment labels, and transformer-based predictions, forming a multi-perspective basis for sentiment labeling and analysis.

Unlike traditional bag-of-words or TF-IDF representations, where preprocessing steps (e.g., stop-word removal, stemming) directly alter the feature space, the embedding model used in this study (Qwen3-Embedding-0.6B) operates on semantic context and is inherently robust to high-frequency function words and minor surface-level noise. Consequently, the preprocessing pipeline primarily targets structural artifacts (e.g., HTML tags, duplicates, missing values) that could distort semantic representations rather than lexical features already captured by the embedding model.

The dataset was further enriched with semantic representations generated by recent embedding models to capture the nuanced sentiment expressed in drug reviews. Unlike traditional bag-of-words or lexicon-based features, embeddings map text into dense vector spaces that preserve semantic similarity and contextual meaning. This representation is particularly important in healthcare, where reviews often contain subtle linguistic patterns, domain-specific terminology, and mixed opinions that cannot be adequately captured with traditional rule-based text methods.

Therefore, several embedding models were systematically benchmarked, including Bge-large-en-v1.5, Qwen3-Embedding-0.6B, Gist-small-Embedding-v0, Gist-Embedding-v0, and Gist-large-Embedding-v0, each offering distinct trade-offs between accuracy and computational cost. Among them, Qwen3-Embedding-0.6B achieved the highest accuracy and F1 scores across multiple evaluation scenarios, demonstrating a strong ability to capture semantic distinctions in patient-generated reviews, in line with results from the Massive Multilingual Text Embedding Benchmark (MMTEB) [24]. In addition to performance, model size and inference speed were critical factors, as they directly impact scalability. As shown in Table 3, Qwen3-Embedding-0.6B achieved the best balance between accuracy and computational cost among the evaluated models, making it a practical choice for processing large-scale review corpora.

Table 3. Embedding model and Random Forest accuracy comparison.

Embedding Model	Parameters (M)	Dimension	Random Forest Accuracy (%)
Qwen3-Embedding-0.6B	595	1024	96.45
Gist-large-Embedding-v0	335	1024	93.46
Bge-large-en-v1.5	335	1024	92.19
Gist-Embedding-v0	109	768	91.91
Gist-small-Embedding-v0	33	384	90.80

3.3. Sentiment Labeling Algorithm

A central challenge in sentiment analysis of drug reviews is the frequent contradiction between numerical user ratings and the textual content of reviews. Representative examples of such contradictions are presented in Table 4. Users may assign a high rating while describing severe side effects or give a low rating despite expressing satisfaction with efficacy. If sentiment labels are derived solely from numerical ratings, these inconsistencies can produce uncorrelated or misleading labels, reducing dataset reliability and model performance. This study addresses this challenge by proposing a structured labeling system that reconciles textual sentiment with rating categories through a rule-based decision framework, ensuring that labels better reflect the true sentiment expressed in patient narratives.

Table 4. Sample reviews with contradiction between numerical user ratings and the textual content of reviews.

Drug Name	Review	Rating	Rating Cat.	Algorithm Label	Label Source	Contradiction
Nitrofurantoin	One more day! I hope the infection don’t come back, because This antibiotic make me feel so sick, now I’m noticing vaginal discharge, I don’t like it at all but this is a side effect.	9	Positive	Negative	Fallback to GPT-5.2	Positive vs. Negative
Linzess	Took Linzess for one month. Side effect is diarrhea so I stayed close to bathroom. Diarrhea tapered off. On day 31 diarrhea hit hard all day. I don’t like the severe diarrhea. I skipped one dose and bowel movement was normal.	10	Positive	Negative	Majority Voting	Positive vs. Negative
Trimethoprim	This is the best medicine as it never ceases to response in extreme cases and takes 4 weeks to resolve the issue.	1	Negative	Positive	High BART Confidence	Negative vs. Positive
Fluoxetine	I’ve been on this med for a year and it works wonders for me, gives me Energy And the drive to do things more, I’ve also lost weight and feels good.	1	Negative	Positive	High BART Confidence	Negative vs. Positive

The proposed labeling mechanism is a hybrid decision framework designed to generate reliable sentiment labels by harmonizing user ratings with multiple automated classifiers. It operates through a layered sequence of decision steps that integrate user input, traditional NLP, and large language models, as detailed below and illustrated in Table 4.

Specifically, the process begins by directly accepting the BART classifier’s output when it aligns with the user’s rating, capitalizing on agreement between human annotation and machine inference. When disagreement occurs, the algorithm applies a confidence filter, adopting BART predictions only when BART’s confidence exceeds a defined threshold (≥0.8). When users explicitly assign a “Neutral” rating, this preference is preserved to maintain user-centric labeling. For reviews where BART confidence is moderate (0.6–0.8), the system examines the degree of agreement across BART, the user rating, and TextBlob outputs. If all three disagree, the user’s rating is favored. Otherwise, the algorithm applies majority voting to determine the most supported category. In cases where BART confidence is low (≤0.6), the decision is escalated to GPT-5.2 [25], which performs zero-shot classification. This final stage leverages the model’s contextual reasoning to resolve ambiguous cases that cannot be reliably addressed by lighter methods.

The proposed framework combines deterministic rules with model-based classification. Unlike prediction-level ensemble methods, which fuse classifier outputs at inference time, it operates upstream by aggregating heterogeneous annotation signals to produce a single label prior to model training. In contrast to black-box approaches, it incorporates explicit decision logic while selectively leveraging generative AI only in cases of high ambiguity. This design aims to reduce annotation noise, resolve inter-model disagreement, and preserve decision provenance.

The system’s label quality is independently validated through both human annotation and LLM-based evaluation. These properties are particularly important in healthcare and pharmacovigilance, where explainability and trustworthiness are essential. Sentiment classification of patient reviews must capture subjective nuances while remaining auditable, aligning with the recommendations of regulatory bodies such as the U.S. FDA [26] and the World Health Organization [27], which emphasize transparency, traceability, and patient-centered design in AI-assisted healthcare systems. The decision rules of the proposed labeling system are summarized in Table 5.

Table 5. Proposed labeling algorithm.

Stages	Condition	Outcome
1—Exact Match	User rating matches BART category	Accept BART prediction/user’s rating
2—High Confidence	BART confidence ≥ 0.8	Accept BART prediction
3—Neutral Preservation	User rating is Neutral	Accept user’s Neutral label
4—All Disagree	User, BART, and TextBlob all different	Accept user’s rating
5—Partial Agreement	At least two sources agree	Apply majority voting to assign label
6—Low Confidence	BART confidence ≤ 0.6	Escalate to GPT-5.2 for zero-shot classification

The distribution analysis of the assigned labels revealed that the majority (73%) resulted from exact matches between user ratings and the BART classifier. In addition, 11% of the labels preserved the user’s neutral preference, while 8% were determined through fallback to GPT-5.2. The remaining rules contributed to a lesser extent: High BART Confidence (4%), Majority Voting (3%), and All Disagree → User Rating (1%). The complete decision flowchart is presented in Figure 1.

Figure 1. Decision flowchart of the proposed hybrid labeling algorithm, illustrating the six-rule cascade from user–BART consensus to GPT-5.2 fallback.

For reproducibility, the exact prompt template used for the GPT-5.2 zero-shot fallback classification (Stage 6) is provided below:

System: You are a clinical sentiment classifier for patient drug reviews. Follow the label definitions strictly and output only the final label.

User: Classify the sentiment of the following drug review.

Categories:

• Positive: Indicates satisfaction with effectiveness, symptom relief, or overall experience.
• Neutral: Contains mixed, uncertain, or purely descriptive statements without clear positive or negative sentiment.
• Negative: Indicates dissatisfaction due to side effects, lack of effectiveness, or overall negative experience.

Review: {text}

Respond with exactly one word: Positive, Neutral, or Negative. Do not add punctuation or any extra text.

The model was invoked with a temperature of 0.0 and a maximum token limit of 10, ensuring single-word outputs. Log-probabilities were enabled to capture prediction confidence.

Human Annotation Protocol

A stratified random sample of 500 reviews was drawn from the labeled Drugs.com corpus for independent manual annotation to evaluate the extent to which the labels generated by the proposed pipeline accurately reflect sentiment. The sample was enriched with reviews from all six labeling rules, ensuring that both consensus cases and ambiguous escalation cases were represented. Two independent annotators, with domain expertise in clinical text analysis, classified each review as Positive, Neutral, or Negative based solely on its textual content, without access to the pipeline-assigned label or user rating.

Inter-annotator agreement reached a Cohen’s $\kappa$ of 0.70, indicating substantial agreement. Disagreements were resolved through discussion to produce a single reference label per review. This annotated subset is used as a benchmark for evaluating the quality of the proposed labeling framework in Section 4.

3.4. System Architecture and Modeling

The proposed framework is structured around a two-stage methodology encompassing preprocessing and modeling, as depicted in Figure 2. In the first stage, user drug reviews are normalized through standard NLP techniques, including tokenization, punctuation removal, and stop-word filtering. For the automatic labeling of sentiment, the proposed labeling strategy is employed that integrates multiple zero-shot models, namely BART (Facebook/bart-large-mnli), TextBlob sentiment scores, review star ratings, and GPT-5.2 sentiment responses. Each model contributes to a composite sentiment label (Positive, Neutral, or Negative), which is then paired with its corresponding cleaned text. In the second stage, the labeled corpus is encoded using the Qwen3-Embedding-0.6B model to capture semantic representations, which are subsequently passed to a Random Forest classifier [28] for supervised training and inference. This modular architecture allows for scalable integration of GenAI models in both labeling and representation learning pipelines.

Figure 2. End-to-end system architecture showing the preprocessing, hybrid labeling, Qwen3-Embedding-0.6B encoding, and Random Forest classification pipeline.

In the experimental stage, multiple machine learning classifiers were systematically evaluated. The selected machine learning models represent diverse learning paradigms, allowing a comprehensive performance assessment. Linear SVM and Logistic Regression provide strong linear baselines suitable for high-dimensional text embeddings, whereas Random Forest and XGBoost [29] capture non-linear relationships and interaction effects between semantic features. LightGBM [30] was included due to its efficiency on large datasets with mixed feature importance, and the Multi-Layer Perceptron enabled evaluation of a neural architecture. K-Nearest Neighbors served as a distance-based baseline using cosine similarity to exploit the semantic embedding space. This diverse model selection ensured a fair and robust comparison across linear, non-linear, distance-based, and neural approaches, establishing a solid foundation for subsequent performance evaluation.

Moreover, hyperparameter optimization was performed for all classifiers. A stratified subset of the data was employed to accelerate the optimization process while preserving class balance. Optuna was used as the hyperparameter optimization framework due to its use of sequential model-based optimization, which enables efficient exploration of conditional and high-dimensional search spaces [31]. In contrast to grid or random search, Optuna evaluates hyperparameter configurations based on probabilistic modeling of past experiments, thereby enabling informed selection of subsequent candidate configurations. The framework also incorporates a pruning mechanism to terminate underperforming experiments based on intermediate validation results, thereby reducing computational overhead during training. This approach is aligned with prior work demonstrating the effectiveness of Bayesian optimization for hyperparameter tuning in machine learning systems [32].

Each classifier was tuned within a problem-specific search space, with selection based on 10-fold cross-validated F1 scores to account for potential class imbalance. Specifically, 20 trials per model were executed using Optuna’s Tree-Structured Parzen Estimator (TPE) sampler with median pruning. This configuration was chosen to balance exploration breadth with computational cost. The search spaces were defined as follows:

• Random Forest: n_estimators from 50 to 300, max_depth from 5 to 20, min_samples_split from 2 to 10, and max_features set to sqrt, log2, or None.
• XGBoost: n_estimators from 50 to 300, learning_rate from 0.01 to 0.3 on a log-uniform scale, max_depth from 3 to 10, subsample from 0.5 to 1.0, colsample_bytree from 0.5 to 1.0, and min_child_weight from 1 to 10.
• LightGBM: n_estimators from 50 to 300, learning_rate from 0.01 to 0.3 on a log-uniform scale, num_leaves from 20 to 150, max_depth from 3 to 20, and min_child_samples from 5 to 30.
• MLP: hidden_layer_sizes set to (50), (100), (50,50), (100,50), (256,128), or (128,64), alpha from ${10}^{-5}$ to ${10}^{-2}$ on a log-uniform scale, learning_rate_init from ${10}^{-4}$ to ${10}^{-1}$ on a log-uniform scale, and activation set to relu, tanh, or logistic.

The remaining classifiers (Logistic Regression, Linear SVM, and KNN) were configured with standard hyperparameters validated through preliminary experiments. A stratified subset of 20,000 samples was drawn from the training set, preserving the original class distribution, to accelerate the optimization process. The selected hyperparameters presented in Table 6 enhanced the performance of the models investigated in detecting different aspects of sentiment across reviews.

Table 6. Selected hyperparameters.

Machine Learning Model	Hyperparameters
Logistic Regression	max_iter = 1000, C = 1.0, solver = ‘saga’, multi_class = ‘multinomial’
Linear SVM	C = 1.0, max_iter = 1000
Multi-Layer Perceptron (MLP)	hidden_layer_sizes = (256, 128), activation = ‘relu’, alpha = 1 $\times {10}^{-5}$, solver = ‘adam’, learning_rate = ‘adaptive’, max_iter = 1000
K-Nearest Neighbors (KNNs)	n_neighbors = 7, metric = ‘cosine’
Random Forest (RF)	n_estimators = 200, max_depth = 20, min_samples_split = 10, min_samples_leaf = 1
XGBoost (XGB)	use_label_encoder = False, eval_metric = ‘mlogloss’, learning_rate = 0.05, max_depth = 7, n_estimators = 200
LightGBM	num_leaves = 64, max_depth = 15, learning_rate = 0.05, n_estimators = 200

A standardized evaluation protocol was applied to ensure a fair comparison across classifiers. All models used identical input representations and preprocessing pipelines, isolating performance differences attributable solely to the learning algorithm. Robustness was further assessed through repeated stratified resampling, reducing dependence on any single data partition and providing a more stable estimate of generalization performance.

Computational Cost

The end-to-end pipeline was executed on a workstation equipped with a high-performance multi-core CPU and 64 GB RAM. Text preprocessing and TextBlob annotation of the full 213,869 review corpus were completed in approximately 15 min. BART zero-shot inference required approximately 8 h on a single NVIDIA A100 GPU. GPT-5.2 classification, applied only to the 8% of reviews escalated via Rule 6 (approximately 17,100 reviews), consumed roughly 3 h of API wall-clock time with 10 concurrent threads. Qwen3-Embedding-0.6B encoding of the full corpus required approximately 4 h on the same GPU. Hyperparameter optimization via Optuna (20 trials per model across 7 classifiers on a 20,000-sample stratified subset) completed in under 2 h. Final 10-fold cross-validation training and evaluation of all classifiers required approximately 1 h. In total, the pipeline runs in under 19 h, with GPT-5.2 API calls representing the only recurring monetary cost. The modular design allows individual stages to be cached and reused when only downstream components are modified.

4. Results

4.1. In-Domain Evaluation

A reliable evaluation of the different sentiment classification algorithms was conducted using a 10-fold stratified cross-validation procedure instead of a simple train–test split. This approach ensures that, in each fold, the class distribution (Positive, Negative, and Neutral) is preserved, maintaining the ratio of sentiment classes observed in the original dataset. This method reduces the risk of bias due to imbalanced data and provides a more robust estimate of the classifiers’ generalization ability. Formulas applied in this study are defined as follows:

$$\begin{array}{cc}\hfill \mathrm{Accuracy}& ={\displaystyle \frac{TP+TN}{TP+FP+TN+FN}}\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill \mathrm{Recall}& ={\displaystyle \frac{TP}{TP+FN}}\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill \mathrm{Precision}& ={\displaystyle \frac{TP}{TP+FP}}\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill \mathrm{F}1\mathrm{Score}& ={\displaystyle \frac{2·\mathrm{Precision}·\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}\hfill \end{array}$$

(4)

where the following are defined:

• TP (True Positive): A review predicted as belonging to a sentiment class when it is indeed annotated as such.
• TN (True Negative): A review correctly predicted as not belonging to a particular sentiment class.
• FP (False Positive): A review incorrectly predicted as belonging to a sentiment class when it does not.
• FN (False Negative): A review belonging to a sentiment class but misclassified as another.

Table 7 presents the quantitative evaluation results. As shown in Figure 3, a clear performance stratification emerges across the seven classifiers. The tree-based models (RF, XGB, and LightGBM) form the top tier, all exceeding 94% accuracy, with RF achieving the highest overall scores (96.45% accuracy and 95.68% macro-F1). A notable performance gap of approximately 4 percentage points separates this top tier from the MLP (90.69%). The MLP, in turn, substantially outperforms the linear models (Logistic Regression and Linear SVM, both near 80%), as well as the distance-based KNN (75.27%).

Table 7. Sentiment classification evaluation (top-three models).

Model	Features	Accuracy (%)	Average	Recall (%)	Precision (%)	F1 (%)
RF	1024	96.45	Weighted	96.45	96.45	96.45
			Macro	95.45	95.91	95.68
XGB	1024	95.10	Weighted	95.10	95.09	95.09
			Macro	93.55	94.24	93.89
LightGBM	1024	94.59	Weighted	94.59	94.60	94.59
			Macro	93.19	93.57	93.37

Results for all seven classifiers are shown in Figure 3. Bold text indicates the results of the proposed approach.

Figure 3. Comparative bar chart of sentiment classification performance across seven machine learning models, evaluated on accuracy, macro-recall, macro-precision, and macro-F1 score.

4.1.1. Per-Class Analysis and Confusion Matrix

A more granular view of classifier behavior is obtained through per-class precision, recall, and F1 scores computed from cross-validated predictions for the top-performing model (Random Forest). Table 8 reports these metrics.

Table 8. Per-class metrics for Random Forest (10-fold cross-validation).

Class	Precision (%)	Recall (%)	F1 (%)	Support
Negative	97.67	95.50	96.57	62,962
Neutral	93.44	93.00	93.22	31,662
Positive	96.61	97.87	97.24	119,245
Macro Avg.	95.91	95.45	95.68	213,869

The per-class results show that the Positive and Negative classes achieve F1 scores above 96%, while the Neutral class, despite being the smallest (14.8% of data), still attains 93.22% F1, with precision and recall closely balanced. The slightly lower Neutral performance reflects the inherent ambiguity of reviews expressing mixed sentiment, which is consistent across all three classifiers. The confusion matrices for the top-three classifiers are presented in Table 9. The dominant diagonal values confirm that the vast majority of predictions align with the assigned labels, while the most frequent confusion occurs between Negative and Positive, consistent with the inherent ambiguity of borderline reviews where mixed side-effect descriptions alongside reported therapeutic benefit make categorical assignment difficult.

Table 9. Confusion matrices for top-three classifiers (10-fold cross-validation).

Model	Actual	Negative	Neutral	Positive
RF	Negative	60,129	340	2493
	Neutral	621	29,443	1598
	Positive	813	1727	116,705
XGB	Negative	59,499	519	2944
	Neutral	1032	28,222	2408
	Positive	1252	2325	115,668
LightGBM	Negative	58,869	614	3479
	Neutral	991	28,359	2312
	Positive	1461	2713	115,071

4.1.2. Representative Failure Cases

A qualitative analysis of representative failure cases was conducted to examine the remaining classification errors. Table 10 presents selected misclassified reviews from the Random Forest cross-validation predictions.

Table 10. Representative failure cases (Random Forest cross-validation).

Review (Excerpt)	True Label	Predicted	Label Source
I have been on this medication for three months. It completely cleared my symptoms, but the nausea and dizziness are unbearable. I am not sure I can continue.	Negative	Positive	GPT-5.2 Fallback
It works okay I guess. Nothing amazing but it does the job. I have not noticed any major side effects so far.	Neutral	Positive	Majority Voting
Great pill! Made me gain 20 pounds and I cannot sleep at night. Absolutely wonderful experience.	Positive	Negative	Exact Match

The error analysis reveals three recurrent patterns. First, reviews that combine therapeutic benefit with severe side effects (Case 1) generate conflicting semantic signals, which the classifier resolves in favor of dominant positive efficacy cues. Second, understated or hedged language (Case 2) lacks the strong lexical markers associated with neutrality, causing the model to default to a mildly positive prediction. Third, sarcastic or ironic reviews (Case 3) employ overtly positive surface tokens while conveying negative intent, a well-documented challenge in sentiment analysis. These cases represent the inherent difficulty of the three-class sentiment task rather than systematic pipeline failure.

4.1.3. Labeling Rule Ablation Study

An ablation study was conducted to assess the contribution of each rule in the hybrid labeling cascade: Each rule was individually removed, and the Random Forest classifier was retrained on the resulting labels. Table 11 summarizes the results.

Table 11. Labeling rule ablation study (Random Forest, 10-fold CV).

Configuration	Accuracy (%)	Macro F1 (%)
Full Pipeline (All Rules)	96.45	95.68
Without Rule 1 (Exact Match)	95.90	95.88
Without Rule 2 (High BART Confidence)	95.46	95.37
Without Rule 3 (Neutral Preservation)	97.28	97.28
Without Rule 4 (All Disagree → User Rating)	96.18	96.17
Without Rule 5 (Majority Voting)	96.10	96.08
Without Rule 6 (GPT-5.2 Fallback)	96.16	96.09

The ablation results validate the design of the full six-rule cascade. Removing Rules 1 and 2 produces the largest performance degradation (−0.55 pp and −0.99 pp accuracy, respectively), confirming that exact-match agreement and high-confidence BART predictions are the most critical labeling signals. Although removing Rule 3 (Neutral Preservation) yields a marginal accuracy increase of 0.83 pp, its inclusion preserves the correct identification of genuinely neutral reviews. This aligns with the requirements of pharmacovigilance, where neutral sentiment (e.g., “no change in symptoms”) carries distinct clinical significance and supports a more patient-centric labeling framework that prioritizes faithful representations of patient-reported experiences. Rules 4–6 show limited individual impact (≤0.35 pp), yet their combined contribution is essential for resolving the remaining edge cases that would otherwise default to unreliable fallback labels. Collectively, the ablation study confirms that no rule is redundant: each addresses a distinct source of labeling ambiguity, and the full pipeline represents the optimal configuration when both classification accuracy and label clinical validity are jointly considered.

4.1.4. BART Confidence Threshold Sensitivity

The BART confidence thresholds (≥0.8 for high confidence; ≤0.6 for low confidence) were initially selected and subsequently validated through a systematic sensitivity analysis, assessing their impact on classification performance. The thresholds were varied across a grid of values, and the labeling pipeline was re-executed for each pair. The resulting labels were used to retrain the Random Forest classifier under identical conditions. Table 12 reports the results.

Table 12. BART threshold sensitivity analysis (Random Forest, 10-fold CV).

High Threshold	Low Threshold	Accuracy (%)	Macro F1 (%)	Labels Changed
0.70	0.50	95.82	95.76	11,367
0.75	0.55	96.14	95.70	5311
0.80	0.60	96.45	95.68	—
0.85	0.65	96.23	95.18	4515
0.90	0.70	95.84	95.77	8239

Bold values indicate the proposed threshold configuration that yields the best overall performance.

The sensitivity analysis confirms that the selected thresholds (0.80/0.60) yield the highest classification performance among all tested configurations, achieving 96.45% accuracy and 95.68% macro-F1. As shown in Table 12, accuracy remains above 95.7% across all five threshold pairs, with a total spread of only 0.63 percentage points, demonstrating that the framework is inherently robust to threshold selection. Relaxing the thresholds to (0.70/0.50) reduces accuracy to 95.82% because overly permissive confidence boundaries allow uncertain BART predictions to bypass the multi-source reconciliation cascade while simultaneously reclassifying 11,367 reviews (5.3% of the corpus) and introducing label instability. Conversely, tightening the thresholds to (0.90/0.70) reduces accuracy to 95.84% by forcing 8239 reviews into unnecessary GPT-5.2 escalation, increasing computational cost without benefit. The configuration (0.80/0.60) emerges as the optimal operating point, achieving the best accuracy while maintaining full label stability and avoiding unnecessary variability across threshold settings.

4.1.5. Independent Human and LLM-as-a-Judge Validation

An independent assessment of whether the labels produced by the hybrid pipeline faithfully reflect human judgment was conducted using the annotated reference subset described in Section 3.3 as an external validation set. The agreement between the gold-standard labels and the hybrid pipeline labels was measured at 0.92 (Cohen’s $\kappa$), corresponding to a 95.0% exact match. This level of concordance confirms that the proposed labeling framework produces labels that are well-aligned with human judgment. The per-rule breakdown revealed that reviews labeled via exact match (Rule 1) exhibited the highest agreement (98.0%), while those escalated to GPT-5.2 (Rule 6) showed slightly lower but still substantial agreement (91.7%), confirming the value of the LLM fallback for genuinely ambiguous cases.

As a complementary external validation, the full set of pipeline-generated labels was compared against independent GPT-5.2 zero-shot classifications under an LLM-as-a-judge protocol [33]. The pipeline achieved over 85% concordance with GPT-5.2 judgments across the entire corpus, confirming alignment with current generative reasoning while preserving the transparency and traceability afforded by the explicit rule-based decision cascade.

4.1.6. Contextualization Within Prior Work

Earlier studies have typically relied on numerical ratings or lexicon-based annotations as ground truth, both of which are susceptible to inconsistencies and noise. For example, Vijayaraghavan and Basu [6] reported 93.85% accuracy using an ANN trained on user ratings, while Colón-Ruiz and Segura-Bedmar [7] achieved 90.46% with a hybrid BERT + LSTM architecture. Similarly, Garg [9] employed LinearSVM with rating-based polarity, reporting 93% in binary classification, and Lee et al. [11] obtained 96% using TextBlob-derived sentiment labels combined with a CNN-LSTM model.

In contrast, the present study shifts the focus from model architecture to the quality of the labels that drive training. The proposed hybrid labeling framework integrates multiple annotation signals (ratings, lexicon-based methods, transformer outputs, and GPT-5.2 inference) within a structured decision process, producing higher-fidelity ground truth before any classifier is applied. When trained on these hybrid labels, a standard Random Forest classifier achieves 96.45% accuracy in three-class classification.

As shown in Table 13, the primary distinction lies not in the reported accuracy values but in the ground-truth construction. Earlier studies rely on single-source annotations (e.g., overall ratings or TextBlob polarity), whereas the proposed framework integrates multiple heterogeneous signals into a unified label. This transition from single-source supervision to multi-signal hybrid labeling constitutes the central methodological contribution of this work.

Table 13. Overview of labeling strategies and classification results across studies.

Ref.	Year	Ground Truth	Model	Classes	Text Representation	Results
[6]	2020	Overall rating	ANN	3	TF-IDF	93.85%
[7]	2020	Overall rating	BERT + LSTM	3	Word2Vec embeddings	90.46%
[9]	2021	Overall rating	LinearSVM	Pos. vs Neg.	Bag-of-Words, TF-IDF, Word2Vec	93%
[11]	2023	TextBlob Sentiments	CNN-LSTM	3	Union of TF-IDF and Bag-of-Words	96%
Current Study	2026	Proposed Labeling Algorithm	RF	3	Qwen3-Embedding-0.6B	96.45%

Direct numerical comparison across studies remains inherently limited, as differences in class definitions, preprocessing pipelines, and dataset versions affect reported performance. Accordingly, Table 13 is intended to contextualize the evolution of labeling strategies rather than to establish a strict performance ranking.

4.1.7. Statistical Significance and Training Data Size Effect

The McNemar test [34], applied over the predictions of all classifiers, confirmed that the superiority of Random Forest over the remaining algorithms was statistically significant. Beyond model selection, dataset size is a significant concern in the medical field, particularly when small training datasets lead to unreliable classification models. The effect of training data size on overall classifier performance was therefore investigated by training models on progressively larger subsets of the available records.

For the purposes of this analysis, the dataset was first separated into training and test splits, and five stratified subsets of increasing size were drawn from the training set. While increasing training data typically improves the signal-to-noise ratio, this benefit diminishes when the training and test sets follow different distributions. In such cases, adding more data from a mismatched distribution may introduce noise rather than reduce it. Ideally, test performance should improve or remain stable with more data, but in no case should it deteriorate significantly.

The results of the investigation showed that the overall performance of the classifiers depends on how much a dataset represents the original distribution and not on its size, consistent with the findings of [35]. Importantly, performance variations across subsets remained small, confirming the absence of overfitting.

4.2. Cross-Source Comparison Between Datasets

The robustness and generalizability of the proposed labeling framework were further assessed by incorporating a second open-source dataset from DrugLib.com, hosted on the UCI Machine Learning Repository [19,36]. This dataset consists of 4091 user reviews and follows a similar structure, enabling cross-dataset evaluation and comparative analysis of model performance.

Prior to model training, a consistency assessment of the hybrid labeling pipeline was conducted to verify the reliability of label generation within the newly integrated dataset. The distribution analysis of the assigned labels revealed that the majority of the sentiment labels (67.3%) were assigned through the exact match rule, indicating a high level of agreement between the user-provided numeric ratings and the sentiment predicted by the BART classifier. An additional 17.5% of the reviews were categorized under User Neutral Preference, thereby preserving the users’ explicitly neutral stance, while 7.4% required fallback to GPT-5.2 to resolve discrepancies between textual and numerical sentiment cues. The remaining rules contributed marginally: High BART Confidence (4.8%), Majority Voting (2.2%), and All Disagree → User Rating (0.8%). This confirms that the hybrid labeling mechanism remained internally balanced and invoked fallback procedures only in cases of elevated ambiguity.

Subsequent cross-source experiments evaluated the transferability of sentiment models trained on the hybrid-labeled Drugs.com corpus to the DrugLib.com dataset and vice versa. Although both sources address user experiences with medications, they differ substantially in linguistic style, corpus size, and structure. Drugs.com reviews are longer and narrative, frequently combining multiple aspects such as efficacy, dosage, and side effects within a single text, whereas DrugLib.com entries are shorter and aspect-specific.

Gräßer et al. [19] reported cross-data sentiment transfer accuracies of 70–75% between Drugs.com and DrugLib.com, with Cohen’s $\kappa$ values below 0.50. The proposed framework achieved comparable cross-source results: When trained on Drugs.com and evaluated on DrugLib.com, the model attained 71.4% accuracy and 67% macro-F1, confirming the portability of the hybrid labels across independently collected corpora. This transferability is attributed to the hybrid labeling algorithm, which enforces semantic consistency between textual sentiment and user-assigned ratings, and to the contextual embeddings generated by the Qwen3-Embedding-0.6B model, which effectively mitigate stylistic and lexical discrepancies between data sources. Furthermore, even when the training corpus was limited to the smaller DrugLib.com dataset, the model maintained an average accuracy of 62.8%, demonstrating bidirectional portability across corpora of different scales.

These findings demonstrate that label fidelity and semantic alignment play a more decisive role in cross-source generalization than dataset scale alone. By integrating GenAI-powered hybrid labeling with contextual embeddings, these results support intra-domain transfer, confirming the portability of the hybrid labels and emphasizing the importance of embedding uniformity and label coherence for stable sentiment modeling across heterogeneous yet semantically aligned corpora.

4.3. Impact of Label Quality on Sentiment Classification

The influence of label quality on sentiment classification performance was examined through a comparative evaluation between the traditional rating-based labeling approach and the proposed hybrid labeling framework. Both methods were applied independently to the Drugs.com and DrugLib.com datasets under identical preprocessing, embedding, and classifier configurations to isolate the effect of labeling quality from other experimental factors.

The results demonstrate that the hybrid labeling framework consistently outperformed the conventional rating-based approach across both datasets. On the Drugs.com corpus, the Random Forest classifier achieved 96.45% accuracy and 95.68% macro-F1, compared to 94.62% accuracy and 94.27% macro-F1 using rating-based labels. A similar improvement was observed on the DrugLib.com dataset, where the hybrid-labeled configuration achieved 80.26% accuracy and 79.66% F1, outperforming the rating-based baseline (75.32% accuracy; 74.85% F1). These gains of approximately +2–5% across distinct data sources confirm that label fidelity and semantic coherence are decisive factors in downstream model performance, independent of the underlying architecture, as summarized in Table 14.

Table 14. Impact of label quality on sentiment classification performance.

Label Type	Dataset	Classifier	Accuracy (%)	Macro-F1 (%)
Rating-based	Drugs.com	RF	94.62	94.27
Hybrid-labeled	Drugs.com	RF	96.45	95.68
Rating-based	DrugLib.com	RF	75.32	74.85
Hybrid-labeled	DrugLib.com	RF	80.26	79.66

By integrating semantic agreement, classifier consensus, and GPT-based fallback reasoning, the hybrid framework effectively mitigates annotation noise and enhances the reliability of sentiment supervision across heterogeneous review corpora.

5. Discussion

This study introduced a data-centric framework for sentiment analysis of drug reviews, developing a hybrid labeling system that reconciles user ratings, lexicon-based outputs, transformer-based classification, and GPT-5.2 zero-shot inference within a structured decision framework. The proposed methodology directly addressed the persistent problem of label inconsistency that has limited prior work in this domain [17]. Coupled with Qwen3-Embedding-0.6B embeddings for semantic representation and Random Forest [28] for supervised classification, the system achieved 96.45% accuracy and 95.68% macro-F1 score, statistically outperforming all comparison algorithms, as confirmed by the McNemar test [34]. Importantly, the gold-standard validation confirmed that the pipeline-generated labels align well with independent human annotation (0.92 Cohen’s $\kappa$), mitigating the risk of circular evaluation.

A central finding of this study is that label quality exerts a stronger influence on classification performance than the choice of learning algorithm. The comparative evaluation demonstrated that switching from rating-based to hybrid labels while keeping the classifier, embeddings, and preprocessing identical yielded gains of +1.8% accuracy on Drugs.com and +4.9% on DrugLib.com. This result supports the data-centric AI thesis [17] and extends it to the pharmaceutical sentiment domain, where the gap between user ratings and textual sentiment is particularly pronounced. Furthermore, the distribution analysis revealed that 73% of labels were resolved through direct user–BART agreement, confirming that the framework incurs minimal computational overhead for the majority of instances and escalates to GPT-5.2 only for genuinely ambiguous cases (8%). The labeling rule ablation study further confirmed that each rule in the cascade contributes meaningfully to overall label quality, with Rules 1 and 2 providing the largest individual effects (−0.55 pp and −0.99 pp accuracy, respectively) and Rule 3 (Neutral Preservation) proving essential for maintaining clinical validity despite a marginal accuracy trade-off.

The BART confidence thresholds (≥0.8 for high confidence and ≤0.6 for low confidence) proved robust across all five tested configurations, with a total accuracy spread of only 0.63 percentage points (Section 4.1.4). This narrow variation implies that the labeling cascade is driven primarily by inter-source agreement rather than by a single threshold choice, reducing the risk of overfitting to corpus-specific confidence distributions.

The proposed labeling system demonstrated over 85% concordance with GPT-5.2 judgments when evaluated under an LLM-as-a-judge framework [33], indicating alignment with current generative reasoning while preserving transparency and traceability through explicit rule-based logic. This dual emphasis on performance and explainability is important for healthcare applications, where trustworthiness and accountability are essential [26,27]. Nevertheless, the current study demonstrates a promising sentiment labeling framework rather than a fully validated pharmacovigilance system. Integration into clinical workflows would require additional regulatory validation, domain expert oversight, and prospective evaluation. In addition, the systematic embedding model comparison revealed that Qwen3-Embedding-0.6B outperformed four alternative models by margins of 3.0–5.7% accuracy, confirming that contextual embeddings trained on large multilingual corpora [24] capture semantic distinctions in patient-generated text more effectively than smaller-dimensional alternatives.

Cross-source evaluation further substantiated the robustness of the proposed framework. When trained on Drugs.com and evaluated on DrugLib.com, the model achieved 71.4% accuracy and 67% macro-F1, consistent with the 70–75% range reported by Gräßer et al. [19], confirming that the hybrid labels transfer effectively despite the substantial stylistic and structural differences between the two corpora. Notably, the cross-source accuracy remained competitive without any domain adaptation or fine-tuning, suggesting that the hybrid labeling mechanism produces labels for which their semantic grounding generalizes beyond the corpus on which they were created.

Despite these strengths, several limitations merit discussion. The current evaluation is confined to English-language drug reviews. Extending the framework to multilingual corpora is necessary to establish broader applicability. Additionally, the GPT-5.2 fallback introduces a dependency on proprietary infrastructure, raising practical considerations around reproducibility and operational cost, particularly in resource-constrained clinical settings.

The user-generated review data is also subject to inherent biases that may influence model generalization. Self-Selection Bias: Users who choose to post reviews are not representative of the general patient population; they may be disproportionately motivated by extreme experiences (either very positive or very negative), leading to under-representation of neutral or moderate opinions. Demographic Bias: The reviewing population on English-language platforms such as Drugs.com and DrugLib.com skews toward specific geographic, age, and socioeconomic groups, which may not reflect the diversity of real-world patient populations. Reporting Bias: Users may omit adverse effects that they consider minor or socially sensitive, or conversely, they may over-report side effects for medications with well-publicized safety concerns. These biases are inherent in the data source and cannot be fully mitigated through preprocessing or labeling alone. However, the hybrid labeling framework partially addresses their consequences by cross-referencing multiple sentiment signals rather than relying on a single (potentially biased) rating or lexicon score.

Furthermore, although the cross-source evaluation confirmed label portability, both datasets originate from the same domain. Whether the labeling strategy transfers effectively to adjacent healthcare text sources, such as clinical trial narratives or electronic health records, remains an open question. Finally, while the gold-standard validation confirmed strong alignment with expert human judgment, the annotated sample represents a subset of the full corpus, and larger-scale annotation studies would further strengthen confidence in label reliability.

6. Conclusions

This paper presents a GenAI-powered framework for reliable sentiment labeling of drug reviews, addressing a core challenge in pharmacovigilance: the frequent misalignment between user-assigned ratings and the actual sentiment expressed in review text. The proposed system integrates a six-rule decision cascade that reconciles transformer-based predictions, lexicon-based scores, user ratings, and GPT-5.2 zero-shot inference to produce high-fidelity labels that are both interpretable and auditable. Combined with Qwen3-Embedding-0.6B semantic representations and a Random Forest classifier, the framework provides a structured labeling approach that may inform future integration into drug safety monitoring workflows.

The key contribution of this work is not the implementation of another sentiment classifier but the demonstration that structured, data-centric label generation can improve classification performance without altering the underlying model architecture. The hybrid labeling algorithm achieved 96.45% accuracy and a 95.68% macro-F1 score, outperforming all evaluated baselines. Gains of +2–5% accuracy were observed consistently across two independent datasets when switching from conventional rating-based labels to hybrid labels, confirming that label fidelity is a decisive factor in downstream performance. Gold-standard validation against independent human annotation confirmed strong agreement (0.92 Cohen’s $\kappa$), while the labeling rule ablation study demonstrated that each rule contributes meaningfully to the cascade. The BART confidence threshold sensitivity analysis confirmed that the framework is robust to moderate parameter perturbations. Cross-source evaluation confirmed label portability across independently collected corpora, while 85% concordance with GPT-5.2 under an LLM-as-a-judge protocol confirmed alignment with current generative language understanding. The framework embeds clear decision logic and records decision provenance, properties essential in healthcare environments where explainability and trustworthiness are important.

Future research directions will focus on three areas. First, multilingual corpora will be incorporated to evaluate generalizability across diverse linguistic settings. Second, multimodal sentiment classification will be investigated by integrating signals beyond textual embeddings, such as behavioral and temporal usage patterns, to advance the scope of human behavioral analysis. Third, the replacement of the proprietary GPT-5.2 fallback with open-source large language models will be explored to improve reproducibility and reduce operational costs. Overall, this framework provides a structured and reproducible approach to sentiment-based drug safety monitoring.