Sentiment Classification of Amazon Product Reviews Based on Machine and Deep Learning Techniques: A Comparative Study

Abstract

Sentiment classification plays a crucial role in analyzing customer feedback to identify market trends, enhance product recommendations, and improve customer satisfaction. This study focuses on sentiment analysis of Amazon reviews using two major datasets—Fine Food Reviews and Unlocked Mobile Reviews—which exhibit label imbalance. To address this challenge, both oversampling and undersampling techniques were applied to balance the datasets. Various machine learning (ML) algorithms, including Random Forest (RF), Logistic Regression (LR), Support Vector Machine (SVM), Naïve Bayes (NB), and Gradient Boosting Machine (GBM), as well as deep learning (DL) models such as Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and transformer-based models like RoBERTa, were implemented. After data cleaning and preprocessing, models were trained, and performance was evaluated. The results indicate that oversampling significantly enhances classification accuracy, particularly for the Fine Food dataset. Among ML models, Random Forest achieved the highest accuracy due to its ensemble approach and robustness in handling high-dimensional data. DL models, particularly RoBERTa, also demonstrated superior performance owing to their capacity to capture contextual dependencies. The findings emphasize the importance of data balancing for optimal sentiment analysis and contribute valuable insights toward advancing automated opinion classification in e-commerce applications.

1. Introduction

Comparative studies are a fundamental component of scientific research, allowing researchers to systematically evaluate the performance of models and algorithms across diverse datasets. Such studies provide insights into the strengths and weaknesses of various approaches and help in developing more accurate and efficient techniques [1]. Within the domain of sentiment classification, comparative research plays an essential role in understanding human emotions, which are complex and expressed through diverse linguistic forms. By evaluating multiple models under comparable conditions, researchers can identify which techniques are most compatible with different contexts and can yield more reliable sentiment predictions.

The significance of comparative research has been underscored in several recent studies. Plaza-del-Arco et al. (2024) [2] highlighted the lack of consensus in sentiment classification frameworks, emphasizing the need for unified comparative approaches to improve model robustness. Likewise, Lomas et al. (2024) [3] examined generative models’ ability to express emotion and found significant variation, reinforcing the necessity of comparative studies to determine the most effective algorithms for emotion modeling. Collectively, such research guides the advancement of sentiment classification and contributes to progress in natural language processing (NLP), one of the most active areas in artificial intelligence.

NLP enables machines to understand and generate human language. Its applications include tasks such as Named Entity Recognition (NER), Question Answering, Text Summarization, and Natural Language Generation [4]. One of the most impactful NLP applications is sentiment classification, an automated process of categorizing text into positive, negative, or neutral sentiment. It is widely used by organizations to assess public opinion, improve recommendation systems, and monitor customer satisfaction. As consumers increasingly rely on online reviews when making purchasing decisions, these opinions significantly influence perceptions of products and services. However, with the exponential growth of unstructured text data, efficiently analyzing these reviews becomes a critical challenge, underscoring the importance of developing effective sentiment classification systems.

In this work, we focus specifically on sentiment classification, which is a well-defined subtask of sentiment analysis. While sentiment analysis broadly refers to the computational study of opinions, emotions, and subjectivity in text, sentiment classification aims to assign predefined polarity labels (e.g., positive, neutral, or negative) to textual data in a supervised learning setting. Accordingly, although the two terms are sometimes used interchangeably in the literature, this paper adopts the term sentiment classification to accurately reflect the scope and methodology of the proposed experiments.

Sentiment classification can be approached in several ways. Geyndt et al. (2022) [5] identified four major types: (1) fine-grained classification, which categorizes text by polarity (positive, negative, neutral); (2) emotion detection, which identifies specific emotions such as joy, sadness, or anger; (3) aspect-based classification, which targets particular product or service attributes; and (4) multilingual classification, which extends analysis across different languages. These tasks rely on machine learning (ML) and deep learning (DL) algorithms depending on dataset characteristics and required interpretability.

Recent research has focused on leveraging artificial intelligence techniques to enhance sentiment classification. Taherdoost and Madanchian [6] emphasized the necessity of automating sentiment-related tasks to handle the vast scale of online data efficiently. Similarly, Nichifor et al. [7] demonstrated that sentiment classification enables e-commerce platforms such as Amazon and eBay to design data-driven marketing campaigns and personalized customer experiences, increasing satisfaction and revenue.

Despite its wide applicability, sentiment classification faces several challenges, including sarcasm, negation, and the presence of neutral expressions. For instance, in the sarcastic review, “Oh wow, this phone case is amazing! It only broke after one day,” the apparently positive word “amazing” actually conveys dissatisfaction. Likewise, in “I thought this tea would be amazing, but it’s not good at all,” negation reverses the sentiment. Many traditional models fail to capture such nuances, resulting in misclassifications [8]. This study addresses these challenges through advanced preprocessing techniques that explicitly consider sarcasm and negation, as well as through the use of transformer-based models such as RoBERTa, which are capable of understanding context-dependent semantics.

This research applies a comparative approach that includes both ML algorithms—Random Forest (RF), Logistic Regression (LR), Naïve Bayes (NB), Support Vector Machine (SVM), and Gradient Boosting Machine (GBM)—and DL algorithms—Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and RoBERTa. These models are evaluated on two large Amazon datasets (Fine Food Reviews and Unlocked Mobile Reviews), focusing on their ability to classify sentiment into three categories: positive, negative, and neutral. In addition, the impact of class imbalance is systematically examined using both undersampling and oversampling strategies under identical experimental conditions. While the experiments are limited to English reviews and computationally constrained by the Google Colab environment, they provide valuable insights into optimizing sentiment classification performance across different modeling paradigms.

Despite extensive prior work, several limitations remain in existing sentiment classification studies. Many existing studies [1,9,10,11] evaluate ML or DL models independently, apply different preprocessing pipelines, or rely on a single dataset, making fair comparison difficult. Moreover, the interaction between class imbalance handling strategies and different model families (ML vs. DL) has not been systematically benchmarked under unified experimental conditions [12]. This study is motivated by the need for a controlled, systematic comparison that isolates the impact of preprocessing, sampling strategies, and model architecture on sentiment classification performance.

The objective of this research is to introduce and evaluate NLP-based sentiment classification models that can categorize Amazon product reviews accurately across multiple sentiment classes. Specifically, the study aims to

(1)

assess recent advancements in sentiment classification models,

(2)

examine the influence of dataset quality and balance on performance,

(3)

compare ML and DL architectures under identical experimental conditions, and

(4)

identify limitations and directions for future improvements.

The importance of this work lies in both its scientific and practical contributions. Scientifically, it establishes a unified experimental benchmark for comparative sentiment classification by standardizing preprocessing, sampling, and evaluation across ML and DL techniques. Practically, it provides organizations with evidence-based guidance for selecting sentiment classification models that balance accuracy, robustness, and computational efficiency. Sentiment analysis has become integral in numerous domains—from politics, where it informs campaign strategies [13], to business, where it supports product enhancement and customer retention [14,15].

The key contributions of this study are summarized as follows:

• A unified benchmarking framework enabling controlled, side-by-side comparison of classical ML and modern DL models under identical preprocessing, sampling, and evaluation settings.
• An empirical investigation of class imbalance effects using both undersampling and oversampling strategies across multiple model families.
• The integration of sarcasm- and negation-aware preprocessing into the sentiment classification pipeline.
• A cross-domain evaluation using two large and linguistically diverse Amazon review datasets, providing practical insights for real-world e-commerce applications.

To achieve these goals, the study is guided by three central research questions:

• What is sentiment classification, and what computational principles govern its application to Amazon reviews?
• How do dataset size, selection, and class balancing influence sentiment classification performance?
• Which ML or DL algorithms achieve the highest accuracy under different data conditions, and how can their performance be further improved?

By addressing these questions, the research advances the understanding of comparative sentiment classification methodologies and their practical applications in e-commerce settings. The remainder of this paper is structured as follows: Section 2 reviews related studies on ML and DL methods for sentiment classification; Section 3 describes the datasets, preprocessing methods, feature extraction techniques, and model architectures; Section 4 presents and analyzes experimental results; Section 5 discusses findings and implications. Moreover, Section 5 concludes the study with recommendations for future research.

2. Literature Review

2.1. Introduction to Sentiment Classification

Sentiment classification (often referred to as opinion mining) is the computational task of identifying and categorizing the sentiment expressed in text—typically as positive, negative, or neutral—based on its conveyed polarity. While the broader term sentiment analysis encompasses multiple subtasks (e.g., emotion detection, subjectivity analysis, and aspect-level sentiment), this review focuses specifically on sentiment classification, as this is the task addressed by the experiments in this study.

The definition and scope of sentiment classification vary by application domain. In e-commerce, sentiment classification is commonly applied to customer reviews to assess satisfaction and support product-related decision-making, whereas in social media and political analysis, it may be used to track public opinion or emotional tone. Sentiment classification can be conducted at different levels of granularity, including document-level, sentence-level, or aspect-level analysis [16,17]. Representative definitions emphasizing polarity-based classification are summarized in Table 1.

In general, the importance of sentiment classification has increased substantially with the rapid growth of online platforms where users freely express opinions. Governments, organizations, and businesses increasingly analyze public sentiment to support decision-making, policy evaluation, and customer engagement strategies [18]. Existing methods range from traditional machine learning (ML) techniques to more recent deep learning (DL) approaches, with each class of methods exhibiting different strengths depending on dataset characteristics, feature representations, and class distributions. Aspect-level sentiment classification, for example, provides finer-grained insights by linking opinions to specific product attributes, while document-level classification is more suitable for large-scale opinion mining tasks. Applications span e-commerce, politics, sociology, disaster management, and competitive intelligence [19,20,21,22,23,24,25,26].

Amazon product reviews provide large-scale, real-world datasets with explicit sentiment labels (star ratings), making them highly suitable for supervised sentiment classification and cross-domain generalization studies [27]. For this reason, much of the existing literature—and the present study—focuses on Amazon review datasets when evaluating sentiment classification models. Accordingly, this review summarizes core concepts, representative ML approaches, and DL models that underpin sentiment classification in such settings [28].

Table 1. Sentiment Classification Definitions.

Source	Definition
[29]	Sentiment classification is the process of determining the sentiment or emotional tone in text, especially within unstructured data from social media and customer feedback.
[30]	Automatic representation of ideas, emotions, and subjectivity in text to determine polarity (positive, negative, neutral), often in binary or multi-class settings.
[31]	A method to distinguish subjective textual data, reflecting the growing interest in analyzing online sources like social media.
[32]	Also called opinion mining; uses text analysis and NLP to identify, extract, and categorize subjective information from unstructured text.
[33]	A growing field focused on understanding a person’s emotions, opinions, and perceptions about a subject/entity to determine overall sentiment (positive/negative/neutral).

Table 1. Sentiment Classification Definitions.

Source	Definition
[29]	Sentiment classification is the process of determining the sentiment or emotional tone in text, especially within unstructured data from social media and customer feedback.
[30]	Automatic representation of ideas, emotions, and subjectivity in text to determine polarity (positive, negative, neutral), often in binary or multi-class settings.
[31]	A method to distinguish subjective textual data, reflecting the growing interest in analyzing online sources like social media.
[32]	Also called opinion mining; uses text analysis and NLP to identify, extract, and categorize subjective information from unstructured text.
[33]	A growing field focused on understanding a person’s emotions, opinions, and perceptions about a subject/entity to determine overall sentiment (positive/negative/neutral).

2.2. Key NLP Concepts and Tools

Modern NLP techniques enable machines to transform unstructured text into machine-interpretable representations [34]. A typical sentiment classification pipeline includes data collection, preprocessing, feature representation, model training, and evaluation [35,36]. Effective preprocessing is critical, as noisy, informal, or unstructured text can substantially degrade model performance. Common preprocessing steps include tokenization, normalization, stop-word removal, and noise filtering [37]. Lemmatization is frequently combined with these steps to improve lexical consistency and downstream classification accuracy and is therefore adopted in the present study.

Textual data must then be converted into numerical representations. Two main families of representations are widely used [38]:

• Discrete representations, such as Bag-of-Words, Term Frequency (TF), and Term Frequency–Inverse Document Frequency (TF-IDF), which are sparse but interpretable and effective for linear and tree-based ML models.
• Distributed representations (embeddings), which encode semantic and contextual information in dense vector spaces. These include static embeddings (e.g., Word2Vec, GloVe, FastText) and contextual embeddings derived from transformer-based models such as BERT and RoBERTa [39].

In line with prior work, TF-IDF representations are typically paired with traditional ML models, while embedding-based representations are more suitable for DL architectures that exploit contextual semantics. Model performance is usually evaluated using standard metrics such as Accuracy, Precision, Recall, and F1-score, often under cross-validation. However, data imbalance [12] and representation choice remain persistent challenges that complicate fair comparison across studies.

2.3. Machine Learning Methods for Sentiment Classification

Machine learning (ML) techniques form the foundation of early and still widely used sentiment classification systems. Common ML classifiers include Logistic Regression (LR), Support Vector Machines (SVMs), Naïve Bayes (NB), Random Forests (RFs), and Gradient Boosting Machines (GBMs).

Logistic Regression is computationally efficient and performs well on linearly separable data, but may struggle with complex non-linear patterns [40]. SVM is particularly effective in high-dimensional feature spaces, making it well suited for TF-IDF representations, although it can be sensitive to class imbalance and computationally expensive for large datasets [41]. Naïve Bayes remains popular due to its simplicity and scalability, but its conditional independence assumption may limit performance on correlated features [42]. Ensemble methods such as RF and GBM improve robustness and predictive power, particularly in noisy or heterogeneous datasets, though they often require careful tuning [43].

Most existing ML-based sentiment classification studies focus on binary classification (positive vs. negative). In contrast, the present study adopts a three-class setting (positive, neutral, negative), which more closely reflects real-world review distributions and introduces additional classification complexity. Introducing a neutral class significantly increases task complexity and can alter relative model performance. Furthermore, ML model effectiveness is highly sensitive to preprocessing choices, feature representation, dataset scale, and class distribution, highlighting the importance of controlled experimental design.

To improve readability and comparability,

Table 2. Classification Studies Using ML Methods.

Ref.	Accuracy	Model	Dataset
[44]	87.61%	LR	Fine Food Amazon Reviews (500,000 reviews)
	84.92%	NB
[45]	86.16%	SVM	67,986 cell-phone reviews (locked/unlocked)
	82.53%	RF
	88.47%	LR	Combined cell-phone + random Amazon products (130,978)
	86.88%	NB
	94.02%	SVM
	92.98%	RF
	93.73%	LR
[38]	87.19%	LR	4,000,000 Amazon reviews (various products)
	84.18%	RF
[46]	82.65%	RF	Amazon Electronics (1,689,188 reviews)
	84.62%	SVM
	80.95%	NB
[47]	84.60%	RF	100M reviews across five categories
	82.30%	NB
	85.90%	LR
[48]	82.00%	LR	Banking sector feedback (websites/social)
	86.00%	RF
	89.00%	SVM
	79.00%	NB
[49]	94.00%	SVM	Fine Food Amazon Reviews (500,000)
	93.00%	RF
	76.00%	XGBoost
[8]	84.34%	NB	34,686,770 Amazon reviews (various)
	85.96%	SVM
	87.53%	LR
	85.09%	RF
	68.58%	KNN

summarizes representative ML-based sentiment classification studies ordered consistently by dataset type and reported F1-score or accuracy, facilitating cross-study comparison.

2.4. Deep Learning Methods for Sentiment Classification

Deep learning (DL) models automatically learn hierarchical feature representations that capture complex syntactic and semantic relationships in text. Convolutional Neural Networks (CNNs) are effective at extracting local n-gram features and phrase-level patterns, making them suitable for sentiment classification tasks that benefit from localized context. However, CNNs are limited in modeling long-range dependencies.

Recurrent architectures such as Long Short-Term Memory (LSTM) networks address this limitation by capturing sequential dependencies, although they can be computationally expensive and sensitive to dataset size and diversity. More recently, transformer-based architectures such as BERT and RoBERTa have emerged as state-of-the-art models, leveraging self-attention mechanisms to capture contextual and semantic relationships across entire documents.

Empirical evidence consistently shows that transformer-based models outperform CNN and LSTM architectures, particularly on large-scale and linguistically diverse datasets. RoBERTa has demonstrated strong robustness to contextual ambiguity and informal language. Nevertheless, even transformer models remain sensitive to dataset imbalance and benefit from high-quality, balanced training data.

Table 3. Classification Studies Using DL Methods.

Ref.	Accuracy	Model	Dataset
[50]	87.72%	CNN	IMDB
[51]	82.30%	LSTM	34,686,770 Amazon reviews (various products)
	94.00%	BERT
[45]	86.46%	CNN	67,986 cell-phone reviews
	86.28%	LSTM
	90.00%	BERT
	96.66%	CNN	Combined cell-phone + random Amazon (130,978)
	92.26%	LSTM
	95.17%	BERT
[52]	86.57%	CNN	4,000,000 Amazon reviews (various)
	50.28%	LSTM
[46]	88.93%	BERT	Amazon Electronics (1,689,188)
	87.10%	LSTM	100 M reviews (five categories)
	84.90%	CNN
	89.00%	BERT
[53]	91.20%	LSTM	Amazon Book Reviews (3M)
	96.30%	RoBERTa

summarizes representative DL-based sentiment classification studies, organized to facilitate comparison across model families and datasets.

2.5. Synthesis and Identified Research Gaps

A synthesis of prior research reveals several consistent findings. Amazon product reviews remain among the most widely used datasets due to their scale, diversity, and availability of labeled sentiment indicators. Preprocessing techniques—such as tokenization, normalization, lemmatization, and noise filtering—consistently improve classification performance. TF-IDF remains a strong baseline representation for ML models, while contextual embeddings from transformer architectures dominate DL-based approaches.

Despite these advances, significant gaps remain. Most Amazon-based datasets are heavily skewed toward positive reviews, leading to degraded performance on minority classes. While oversampling and undersampling strategies are commonly applied, more advanced approaches—such as pseudo-data generation and long-tailed learning frameworks—are rarely explored in comparative sentiment classification studies. Recent research has introduced data-centric solutions that aim to improve minority-class representation through synthetic sample generation and multi-view learning strategies rather than simple duplication of existing samples. Recent work, including Trusted Multi-view Learning for Long-tailed Classification [12], highlights promising directions for addressing extreme class imbalance; however, such methods are often computationally intensive or difficult to integrate with large-scale NLP pipelines.

Moreover, many prior studies evaluate ML and DL models separately or under different preprocessing and evaluation settings, making it difficult to draw fair conclusions about their relative performance. This lack of controlled, unified benchmarking represents a key limitation in the existing literature.

Motivated by these gaps, the present study conducts a comprehensive, controlled comparison of representative ML (LR, SVM, RF, NB and GBM) and DL (CNN, LSTM, RoBERTa) models under identical preprocessing, sampling, and evaluation conditions. By explicitly analyzing the effects of class imbalance through oversampling and undersampling across two large Amazon review datasets, this work seeks to identify sentiment classification strategies that balance accuracy, robustness, and scalability in real-world e-commerce environments.

3. Methodology

This section outlines the methodology adopted in this study, encompassing dataset selection, preprocessing procedures, feature representation, learning algorithms, class rebalancing strategies, training configurations, and evaluation protocols. Figure 1 illustrates the overall proposed sentiment classification framework, providing a unified view of the preprocessing pipeline, class balancing strategies, feature extraction mechanisms, and model training stages. The framework standardizes dataset preprocessing, balancing strategies, feature representation, model families, and evaluation metrics to ensure fair benchmarking across machine learning and deep learning architectures. Unlike prior studies that evaluate models in isolation, this framework enables systematic cross-domain and cross-sampling comparison under identical experimental conditions.

The preprocessing pipeline was designed to ensure data quality and model robustness, incorporating explicit handling of linguistic nuances such as negation and sarcasm, which are often overlooked yet critical for accurate sentiment interpretation. The experimental setup was implemented on Google Colab due to its compatibility with diverse machine and deep learning libraries and its support for GPU acceleration, enabling efficient model training and comparison.

The workflow begins with data acquisition and proceeds through systematic stages: (1) text cleaning and normalization, (2) negation and sarcasm handling, (3) dataset partitioning into training, validation, and testing subsets, (4) class rebalancing via oversampling and undersampling, (5) feature extraction using TF-IDF or contextual embeddings, (6) model training using ML and DL architectures, and (7) evaluation and statistical analysis. This end-to-end design ensures methodological consistency and fair benchmarking under identical experimental conditions.

3.1. Datasets

Two publicly available Amazon review corpora were utilized in this study to capture a wide range of domains, writing styles, and linguistic variations. The first dataset, Fine Food Reviews, contains free-form product reviews accompanied by star ratings and is widely recognized as a benchmark in sentiment classification research. The second dataset, Unlocked Mobile Reviews, focuses on customer feedback related to mobile phones and electronic products, providing domain-specific insights into consumer sentiment. For both datasets, the sentiment labels were derived directly from the star ratings, where higher ratings correspond to positive sentiment, lower ratings to negative, and mid-range ratings to neutral. Adopting a three-class classification scheme (positive, neutral, negative) allows for a more nuanced representation of sentiment beyond traditional binary polarity, reflecting real-world variability in customer opinions. While the original datasets include metadata such as review timestamps, helpfulness scores, and summaries, this study focuses exclusively on the review body text for model training and analysis, using the star ratings as gold-standard sentiment labels. This approach ensures consistent labeling and emphasizes the textual content as the primary source of sentiment information.

3.2. Preprocessing Pipeline

Prior to model training, the text data underwent a comprehensive normalization and cleaning process to ensure consistency and reduce noise. Let x represent a raw review input. Each review was first converted to lowercase and normalized using Unicode standards to eliminate inconsistencies in character encoding and diacritics. Non-linguistic elements such as URLs, HTML tags, emojis, and numeric strings were removed, while punctuation marks relevant to sentiment cues and scope detection were preserved. Contractions and common spelling variations were expanded and standardized (e.g., “can’t” → “cannot”), supported by domain-specific lookup tables for colloquial expressions. Tokenization was applied using either wordpiece or whitespace-aware segmentation, depending on the downstream model requirements. Stopwords were filtered using a standard English stopword list, though negation indicators such as “not,” “no,” and “never” were explicitly retained to preserve sentiment polarity. Lemmatization was then performed to reduce inflected forms to their base lemmas using NLTK’s WordNet or spaCy libraries. Finally, reviews below a minimum length threshold were discarded to prevent sparsity and ensure that only substantive textual content contributed to the learning process. This multi-stage preprocessing pipeline ensured the linguistic clarity and uniformity necessary for effective sentiment classification.

This preprocessing strategy was fixed across all experiments to isolate the effects of model architecture and class imbalance handling.

3.2.1. Negation Handling

To mitigate polarity inversion caused by negation structures, a scope-tagging mechanism was implemented during preprocessing. In this approach, tokens appearing within a defined negation window—bounded either by punctuation marks or by a fixed window size (k)—were suffixed with the marker _NEG. For example, the phrase “not good at all” was transformed into “not good_NEG at_NEG all_NEG.” This method enables the model to capture contextual polarity shifts introduced by negation words, which are often overlooked in standard bag-of-words or TF-IDF representations. By encoding negation scope explicitly, the resulting feature set becomes more polarity-aware, enhancing the discriminative power of traditional machine learning classifiers that rely on TF-IDF features for sentiment analysis.

3.2.2. Sarcasm Cues

Sarcasm, which often inverts the literal polarity of statements, was partially addressed through a combination of pattern-based cues and contrastive markers to enhance semantic interpretation. A curated cue lexicon was developed to detect common sarcastic expressions such as “yeah right,” “as if,” or exaggerated praise like “totally amazing” following descriptions of failure or disappointment. Additionally, contrastive conjunctions (e.g., but, though, however) were used to identify polarity reversals where a positive phrase is followed by a negative one—or vice versa. Specific syntactic templates were also utilized, such as “positive adjective + negative event” patterns (e.g., “amazing… broke in a day”), to capture implicit sarcasm. When sarcasm was detected, corresponding segments were either annotated with a special marker (e.g., _SARC) or split into separate clauses to allow linear models, particularly TF-IDF–based classifiers, to better distinguish polarity shifts. Although transformer-based models such as RoBERTa inherently capture many of these contrastive and contextual patterns through self-attention, explicit tagging provided additional benefits in low-resource cases or when sarcasm was subtle or domain-specific.

3.3. Feature Representations

Two types of feature representations were employed, each tailored to the specific learning paradigm.

• TF-IDF for Machine Learning Models: For traditional ML classifiers, features were represented using Term Frequency–Inverse Document Frequency (TF-IDF) vectors. The vectorizer utilized word uni- and bi-grams (ngram_range = (1, 2)) with sublinear term frequency scaling and smoothed IDF weighting. To reduce sparsity, the vocabulary was restricted to the top 30,000–50,000 terms ranked by document frequency, with a minimum frequency threshold (min_df) applied to eliminate infrequent noise terms. Negation-aware features introduced during preprocessing (e.g., tokens ending with _NEG) were treated as independent dimensions to preserve sentiment polarity. The TF-IDF value for a term t in document d is computed as

$$\mathrm{t}\mathrm{f}\mathrm{i}\mathrm{d}\mathrm{f}(t,d)=(1+\log \mathrm{tf}(t,d))\cdot \mathrm{l}\mathrm{o}\mathrm{g}{\displaystyle \frac{N}{1+\mathrm{d}\mathrm{f}\left(t\right)}}$$

• Embeddings for Deep Learning Models: For deep models, token embeddings served as the primary input representation. CNN and LSTM networks were initialized with either randomly learned embeddings or pre-trained vectors such as GloVe. For transformer-based models (RoBERTa), byte-pair encoding (BPE) tokenization was applied, and contextual embeddings were generated through the transformer encoder, fine-tuned end-to-end during training.

3.4. Learning Algorithms

3.4.1. Machine Learning Models

• Logistic Regression (LR): Implemented with L2 regularization using the liblinear or saga solver; class weighting was applied when rebalancing was disabled.
• Support Vector Machine (SVM): Linear kernel SVM optimized for high-dimensional TF-IDF features, trained with hinge loss and C-regularization.
• Naïve Bayes (NB): Multinomial NB with additive (alpha) smoothing.
• Random Forest (RF): Ensemble of bootstrapped decision trees, with maximum depth constraints to reduce overfitting.
• Gradient Boosting (GBM/XGBoost): Gradient-boosted shallow trees (depth 3–6) with adaptive learning-rate scheduling and early stopping.

3.4.2. Deep Learning Models

• CNN: One-dimensional convolutional layers applied over token sequences with multiple kernel sizes (3, 4, 5), followed by max-pooling, dropout, and a dense softmax output layer.
• LSTM: Single or bidirectional LSTM networks with configurable hidden sizes, optional attention mechanisms, and dropout regularization. Although transformer-based architectures currently represent the state of the art in many NLP tasks, LSTM was intentionally included as a sequence-based deep learning baseline. The objective of this study is not to propose a new architecture but to compare representative model families under identical preprocessing and sampling conditions. Including LSTM enables evaluation of recurrent sequential modeling relative to convolutional and transformer-based approaches, allowing analysis of accuracy–efficiency trade-offs and computational cost considerations that remain relevant for resource-constrained deployments [54].
• RoBERTa: Fine-tuned roberta-base model incorporating a classification head, layer-wise learning-rate decay, and warmup scheduling.
  Hyperparameters for all models were optimized using grid or random search on the validation set. Final configurations were reported alongside results.

3.5. Train/Validation/Test Protocol

Data were partitioned using a stratified 70/10/20 split to maintain class ratio consistency. For ML models, 5-fold cross-validation was conducted on the training set to tune parameters such as regularization strength (SVM/LR), number and depth of trees (RF), and learning rate (GBM). For DL models, early stopping based on validation loss and macro F1-score was applied, with the best-performing checkpoint retained. All experiments used fixed random seeds for data splitting and initialization to ensure reproducibility.

3.6. Class Rebalancing

To address label imbalance, rebalancing was applied only to the training set, leaving validation and test sets untouched. Two strategies were used:

• Random Oversampling (ROS): Duplication of minority-class samples.
• Random Undersampling (RUS): Downsampling of majority-class samples.
  Each model was trained under three conditions—original, ROS, and RUS—to assess robustness to imbalance. SMOTE and other synthetic text generation methods were excluded to avoid introducing artificial semantic patterns that may distort contextual representations, particularly for transformer-based models. Advanced imbalance-handling approaches such as pseudo-data generation and long-tailed learning methods were not incorporated in the experimental setup to preserve controlled comparability across model families and to avoid introducing synthetic semantic artifacts that may bias contextual language representations. Instead, classical resampling strategies were selected as standardized baselines for fair benchmarking.

3.7. Computing Environment

All experiments were executed in Google Colab, chosen for its accessibility, GPU acceleration, and integration with Python ML/DL libraries.

• Python version: 3.x
• Core Libraries: pandas, numpy, scikit-learn, imbalanced-learn, nltk, spaCy, matplotlib
• Deep Learning Frameworks: TensorFlow/Keras and PyTorch 3.12/Transformers for RoBERTa fine-tuning
• Hardware: Google Colab CPU and NVIDIA T4 GPU instances

3.8. Evaluation Metrics and Analyses

Given the three-class setting (positive, neutral, negative), performance was evaluated using Accuracy, Macro Precision, Macro Recall, and Macro F1-score. Per-class metrics and confusion matrices were computed to assess individual category performance. ROC-AUC (one-vs-rest or one-vs-one) was included as supplementary analysis, and the detailed ROC-AUC results are provided in the Supplementary Materials.

Formally, for class c:

$${\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}_c={\displaystyle \frac{{\mathrm{T}\mathrm{P}}_c}{{\mathrm{T}\mathrm{P}}_c+{\mathrm{F}\mathrm{P}}_c}},{\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}_c={\displaystyle \frac{{\mathrm{T}\mathrm{P}}_c}{{\mathrm{T}\mathrm{P}}_c+{\mathrm{F}\mathrm{N}}_c}},{\mathrm{F}1}_c={\displaystyle \frac{2\cdot {\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}_c\cdot {\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}_c}{{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}_c+{\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}_c}}$$

Macro-averages were computed as unweighted means across all classes to reflect minority-class contributions. To ensure fairness, preprocessing and feature extraction were held constant within each model family, identical data splits were applied, and significance testing was conducted across repeated runs.

3.9. Reproducibility

Reproducibility was ensured by fixing random seeds, documenting library versions, and employing deterministic preprocessing and vectorization pipelines. Vocabulary lists and tokenizers were stored to guarantee consistency across reruns. Additionally, configuration files specifying hyperparameters, batch sizes, and epochs, along with complete training and evaluation scripts, were made available to facilitate exact experiment replication.

4. Results and Analysis

4.1. Fine Food Dataset (Undersampling)

This section presents and interprets the experimental results obtained from five machine learning (ML) models—Logistic Regression (LR), Random Forest (RF), Naïve Bayes (NB), Support Vector Machine (SVM), and Gradient Boosting Machine (GBM)—and three deep learning (DL) models—Long Short-Term Memory (LSTM), Convolutional Neural Network (CNN), and RoBERTa. All models were trained and tested on two Amazon review datasets, Fine Food Reviews and Unlocked Mobile Reviews, following class balancing procedures using both oversampling and undersampling techniques. The objective of this analysis is to identify the comparative strengths and limitations of each model in sentiment classification tasks and to examine the impact of data balancing strategies on model performance.

Logistic Regression (LR) with TF-IDF features trained on the original (non-rebalanced) training split data was defined as the global baseline configuration. Baseline performance for both datasets is reported in Appendix A (

Table A1. Global Baseline Performance (Logistic Regression + TF-IDF, No Rebalancing).

Dataset	Accuracy	Precision	Recall	F1-Score	ROC-AUC
Fine Food Reviews	0.7815	0.7832	0.7810	0.7820	0.9145
Unlocked Mobiles Reviews	0.8675	0.8698	0.8669	0.8682	0.9585

). All undersampling and oversampling experiments were evaluated relative to this baseline to ensure controlled and transparent performance comparison.

The evaluation was conducted using standard performance metrics, including accuracy, precision (P), recall (R), F1-score, ROC-AUC, and confusion matrices to provide a comprehensive assessment. The following subsections discuss and compare these results in detail, highlighting the most effective models for achieving robust and generalizable sentiment classification across diverse e-commerce domains.

4.1.1. ML Results

Table 4. ML Results for Fine Food Dataset Using Undersampling.

Algorithm	Accuracy	Precision	Recall	F1	ROC_AUC
LR	0.7414	0.7416	0.7418	0.7417	0.8948
NB	0.7222	0.7270	0.7225	0.7243	0.8769
RF	0.6177	0.6221	0.6183	0.6134	0.8060
GBM	0.6559	0.6578	0.6561	0.6567	0.8293
SVM	0.7341	0.7329	0.7347	0.7335	0.8875

presents the results of the ML algorithms applied to the Fine Food Reviews dataset using undersampling. The evaluation metrics include Accuracy, Precision, Recall, F1-Score, and ROC-AUC, while the corresponding confusion matrices are illustrated in Figure 2. All models were trained and evaluated in a Google Colab environment using CPU resources.

Logistic Regression (LR) achieved the highest performance, with accuracy = 0.7414, precision = 0.7416, recall = 0.7418, F1 = 0.7417, and ROC-AUC = 0.8948. Its stability under undersampling demonstrates robustness against class imbalance while maintaining linear decision boundaries with minimal parameters. The SVM model followed closely (accuracy = 0.7341, ROC-AUC = 0.8875), benefiting from its ability to define clear separation margins after class balancing. Both LR and SVM effectively distinguished sentiment classes, offering reliable and interpretable performance.

Conversely, Random Forest (RF) and Gradient Boosting (GBM) showed lower accuracies (0.6177 and 0.6559), primarily due to loss of feature diversity after undersampling. These ensemble models are more memory-intensive and computationally demanding. Naïve Bayes (NB) achieved moderate results (accuracy = 0.7222) but was limited by its independence assumption.

Based on confusion matrices (Figure 2), LR and SVM yielded the best precision and recall across all classes, while RF and GBM misclassified neutral reviews more frequently. LR and SVM were the most effective ML models, balancing accuracy, interpretability, and computational efficiency.

To avoid data leakage and inflated performance estimates, all sampling strategies (undersampling and oversampling) were applied exclusively to the training partition after stratified splitting. Validation and test sets remained untouched throughout the experiments.

4.1.2. DL Results

This subsection reports the performance of the DL models—CNN, LSTM, and RoBERTa—on the Fine Food Reviews dataset under the undersampling setting. CNN and LSTM were trained and evaluated in Google Colab using CPU resources, while RoBERTa was fine-tuned on a GPU due to its higher computational demands. Learning curves (training/validation accuracy and loss) are summarized in Figure 3, Figure 4 and Figure 5, and aggregate confusion matrices are shown in Figure 6.

Table 5. CNN Results for Fine Food Dataset Using Undersampling.

Epoch	Accuracy	Loss	Val Accuracy	Val Loss	Time (s)	Time per Step (ms)
1	0.5964	0.8485	0.7210	0.6457	31	13
2	0.7495	0.5949	0.7227	0.6403	28	13
3	0.8008	0.4934	0.7229	0.6518	28	13
4	0.8385	0.4136	0.7140	0.6991	28	12
5	0.8686	0.3415	0.7082	0.7530	28	12

presents the epoch-wise results for the CNN. Training accuracy increased steadily from 0.5964 (Epoch 1) to 0.8686 (Epoch 5), with training loss decreasing from 0.8485 to 0.3415. In contrast, validation accuracy began at 0.7210 and closed at 0.7082, while validation loss rose from 0.6457 to 0.7530. These diverging curves (Figure 3) indicate overfitting: the network continued to memorize training patterns while validation performance softened. Runtime per epoch remained stable (28–31 s), reflecting consistent computational throughput under the current undersampling setup. The observed divergence between training and validation curves suggests that undersampling may reduce generalization capacity by discarding informative majority-class samples. Regularization (e.g., dropout) and early stopping were applied to mitigate overfitting.

The relatively small number of epochs (3–5) was intentionally selected based on early convergence observed in validation metrics. Preliminary trials with additional epochs (up to 10) did not yield meaningful improvements in macro F1-score but increased the divergence between training and validation loss, indicating overfitting. Therefore, early stopping criteria were applied to preserve generalization.

Table 6. LSTM Results for Fine Food Dataset Using Undersampling.

Epoch	Accuracy	Loss	Val Accuracy	Val Loss	Time (s)	Time per Step (ms)
1	0.3439	1.0987	0.3478	1.0979	347	154
2	0.3486	1.0926	0.3406	1.0980	344	154
3	0.3567	1.0815	0.3391	1.1068	339	152
4	0.3675	1.0562	0.5516	0.9462	343	154
5	0.6403	0.7832	0.7099	0.6912	341	153

shows that the LSTM improved markedly with training: accuracy rose from 0.3439 to 0.6403, and validation accuracy increased from 0.3478 to 0.7099 by Epoch 5. Both training loss (1.0987 → 0.7832) and validation loss (1.0979 → 0.6912) decreased (Figure 4), indicating progressively better fit without pronounced overfitting. The chief trade-off is computational: each epoch required ~339–347 s (≈5–6 min), far slower than CNN due to LSTM’s sequential computations and state maintenance. In this study, LSTM is included as a representative recurrent baseline to evaluate sequential modeling behavior relative to convolutional and transformer architectures under identical experimental conditions, rather than as a state-of-the-art model. Although undersampling reduces the number of examples (and thus memory), it cannot fully offset the intrinsic sequential cost of LSTM.

RoBERTa was fine-tuned for three epochs; results appear in

Table 7. RoBERTa Results for Fine Food Dataset Using Undersampling.

Epoch	Training Loss	Validation Loss	Accuracy	Precision	Recall	F1
1	0.5109	0.5017	0.7905	0.7895	0.7913	0.7886
2	0.4464	0.5050	0.7966	0.8019	0.7968	0.7987
3	0.3282	0.5557	0.7952	0.7952	0.7957	0.7954

. The best validation accuracy and F1 occurred at Epoch 2 (accuracy 0.7966, precision 0.8019, recall 0.7968, F1 0.7987). While training loss steadily decreased (0.5109 → 0.3282) (Figure 5), validation loss increased in Epoch 3 (0.5050 → 0.5557), signaling incipient overfitting. As expected, transformer fine-tuning incurred the highest compute and memory costs due to large parameter counts and full-sequence self-attention.

Table 8. Best Validation Results for Fine Food Dataset Using Undersampling.

Algorithm	Accuracy	Precision	Recall	F1	ROC_AUC
CNN	0.7082	0.7120	0.7086	0.7100	0.8686
LSTM	0.7099	0.7079	0.7105	0.7089	0.8670
RoBERTa	0.7966	0.8019	0.7968	0.7987	-

aggregates the best validation results per model. RoBERTa clearly outperformed LSTM and CNN, reflecting transformers’ strength in modeling long-range dependencies and nuanced context. LSTM slightly edged CNN on validation accuracy/F1 by leveraging sequential structure, whereas CNN trained much faster and was more memory-efficient but was more prone to overfitting and struggled with neutral/negative separation. Confusion matrices (Figure 6) corroborate these trends: RoBERTa delivered the most balanced class-wise predictions; LSTM showed residual confusion between neutral and positive; CNN underperformed on neutral and negative.

From a practical perspective, these findings suggest that lightweight linear models (LR, SVM) remain viable for resource-constrained deployment, whereas transformer-based architectures such as RoBERTa are preferable in environments where computational infrastructure permits deeper contextual modeling.

Takeaway. Under undersampling, RoBERTa delivered the highest accuracy/F1 but at the greatest computational cost. LSTM offered strong generalization with heavy runtime, whereas CNN was fastest but more prone to overfitting and less effective on neutral sentiment. Choice of model therefore hinges on the accuracy–efficiency trade-off: RoBERTa for maximum performance, LSTM when sequential context is critical, and CNN for fast, resource-constrained settings.

Unlike prior studies that evaluate machine learning and deep learning models independently, this work systematically examines how model ranking changes under controlled sampling regimes across heterogeneous domains. Notably, Random Forest exhibits dramatic sensitivity to oversampling in the Fine Food dataset (F1: 0.6134 → 0.9896), while transformer-based models show stronger robustness to linguistic complexity. This domain-dependent performance shift represents a key empirical contribution of this study.

All reported results represent the best validation performance across repeated runs with fixed random seeds to ensure reproducibility and robustness.

4.2. Fine Food Dataset (Oversampling)

This section reports results on the Fine Food Reviews dataset after class rebalancing via oversampling. We present machine learning (ML) and deep learning (DL) findings, analyze confusion matrices, and compare models with respect to accuracy, computational cost, and memory footprint.

Logistic Regression (LR) with TF-IDF features trained on the original (non-rebalanced) training split was used as the global baseline configuration. Baseline performance for both datasets is reported in Appendix A (Table A1). All undersampling and oversampling experiments were evaluated relative to this baseline to ensure controlled and transparent comparison.

4.2.1. ML Results

Table 9. ML Results for Fine Food Dataset Using Oversampling.

Algorithm	Accuracy	Precision	Recall	F1	ROC_AUC
LR	0.8250	0.8265	0.8250	0.8256	0.9431
NB	0.7323	0.7344	0.7324	0.7333	0.8879
RF	0.9896	0.9896	0.9896	0.9896	0.9994
GBM	0.6663	0.6686	0.6663	0.6671	0.8375
SVM	0.8296	0.8306	0.8297	0.8300	0.9436

summarizes the performance of LR, NB, RF, GBM, and SVM trained in Google Colab (CPU). Evaluation metrics include Accuracy, Precision (P), Recall (R), F1, and ROC_AUC. Confusion matrices are shown in Figure 7.

Random Forest (RF) achieves the best overall performance (Accuracy/F1 = 0.9896, ROC_AUC = 0.9994), reflecting the benefits of ensemble learning when class balance is restored. However, the large number of trees increases computational cost and memory usage.

Oversampling was applied strictly to the training partition after stratified splitting, while validation and test sets remained untouched. This prevents data leakage and ensures that the observed performance gains reflect improved class representation rather than artificial duplication across splits.

LR (Accuracy = 0.8250) provides a strong, computationally efficient baseline due to simple linear operations, while SVM attains comparable accuracy (0.8296) at somewhat higher compute cost. NB (0.7323) degrades under oversampling, as probability estimates can be skewed by replicated duplicated features. GBM (0.6663) underperforms; sequential tree growth is sensitive to noisy or redundant synthetic samples introduced by oversampling, raising overfitting risk and resource use.

RF shows outstanding balance across classes, with very high main-diagonal counts (≈60k+ per class) and minimal cross-class errors. LR is competitive but less precise on the negative class. NB exhibits sizable confusion between negative and neutral. GBM struggles most—especially with neutral. SVM is strong on negative/positive, but less so on neutral. Overall, RF is the preferred ML model under oversampling given its accuracy and stable class-wise behavior—budget permitting.

Although oversampling substantially improves classification metrics, it may introduce redundancy and increase the risk of model memorization, especially in ensemble-based architectures. Therefore, performance gains should be interpreted alongside computational cost and potential generalization trade-offs.

4.2.2. DL Results

CNN and LSTM were trained/evaluated in Colab (CPU). RoBERTa was fine-tuned on GPU. Learning curves appear in Figure 8, Figure 9 and Figure 10; confusion matrices are given in Figure 11.

The number of training epochs (5 for CNN/LSTM and 3 for RoBERTa) was selected based on validation convergence behavior. Preliminary experiments with additional epochs produced marginal accuracy gains but increased divergence between training and validation loss, indicating overfitting. Early stopping criteria were therefore applied to preserve generalization performance.

Table 10. CNN Results for Fine Food Dataset Using Oversampling.

Epoch	Accuracy	Loss	Val Accuracy	Val Loss	Time (s)	Time per Step (ms)
1	0.7743	0.5458	0.9126	0.2583	372	16
2	0.9386	0.1870	0.9477	0.1661	367	16
3	0.9689	0.0981	0.9591	0.1362	364	16
4	0.9805	0.0628	0.9647	0.1431	374	16
5	0.9861	0.0448	0.9688	0.1336	372	16

shows that training accuracy improved from 0.7743 (Epoch 1) to 0.9861 (Epoch 5). Validation accuracy rose from 0.9126 to 0.9688, and validation loss fell from 0.2583 to 0.1336, indicating consistent generalization gains. Time/step remained ~16 ms across epochs. Oversampling increased memory consumption (larger dataset), but produced higher validation accuracy and lower validation loss than in the undersampling setting.

Table 11. LSTM Results for Fine Food dataset Using Oversampling.

Epoch	Accuracy	Loss	Val Accuracy	Val Loss	Time (s)	Time per Step (ms)
1	0.4260	1.0020	0.8087	0.4835	209	9
2	0.8538	0.3899	0.9098	0.2663	205	9
3	0.9357	0.1952	0.9415	0.1838	205	9
4	0.9614	0.1227	0.9510	0.1602	205	9
5	0.9711	0.0927	0.9547	0.1525	206	9

indicates a pronounced learning curve: accuracy rose from 0.4260 to 0.9711; validation accuracy improved from 0.8087 to 0.9547; and validation loss declined from 0.4835 to 0.1525. Time/step remained ~9 ms. While LSTM processes sequences (higher state-management overhead), the balanced, larger dataset provided by oversampling enabled strong generalization.

RoBERTa benefited markedly from oversampling (

Table 12. RoBERTa Results for Fine Food Dataset Using Oversampling.

Epoch	Training Loss	Validation Loss	Accuracy	Precision	Recall	F1
1	0.3721	0.2139	0.9327	0.9350	0.9327	0.9330
2	0.1052	0.1431	0.9694	0.9701	0.9694	0.9694
3	0.1041	0.1190	0.9782	0.9787	0.9782	0.9781

). The best epoch (Epoch 3) achieved accuracy 0.9782 and F1 0.9781, with both training and validation losses decreasing steadily (Figure 10). Considering computational cost, Epoch 2 (Accuracy = 0.9694, Validation Loss = 0.1431) may provide a favorable accuracy–efficiency trade-off for practical deployment scenarios.

Table 13. DL Average Results for Fine Food Dataset Using Oversampling.

Algorithm	Accuracy	Precision	Recall	F1	ROC_AUC
CNN	0.9688	0.9691	0.9688	0.9686	0.9942
LSTM	0.9655	0.9660	0.9654	0.9653	0.9935
RoBERTa	0.9782	0.9787	0.9782	0.9781	-

aggregates the best validation metrics. RoBERTa leads (accuracy 0.9782), followed by CNN (0.9688) and LSTM (0.9655). Confusion matrices show that RoBERTa yields the most balanced predictions, with the highest main-diagonal counts (e.g., 60,578, 61,389, 50,979). CNN is strong but shows residual confusion on positive; LSTM is well balanced, particularly strong on positive.

Oversampling substantially improved results across both ML and DL families by correcting class skew. Among ML methods, RF achieved the highest accuracy (98.96%) and near-perfect class balance, albeit with higher memory and compute demands. LR and SVM remained strong, efficient alternatives for resource-constrained deployments. Among DL models, RoBERTa delivered the best accuracy (97.82%) and F1, thanks to transformer-based contextual modeling; its drawback is heavy GPU cost. CNN and LSTM also attained high accuracy with lower resource usage, making them attractive when fast training/inference and moderate hardware are priorities. From a practical perspective, organizations operating in resource-rich environments may prioritize RoBERTa or RF for maximum predictive performance, particularly when class imbalance is severe. Conversely, in real-time or edge-computing scenarios where memory and inference speed are critical, LR, SVM, CNN, and LSTM provide favorable accuracy–efficiency trade-offs.

Compared to the undersampling setting (Section 4.1), oversampling produced substantially higher accuracy and F1-scores across all model families. The most dramatic improvement was observed for RF (61.77% → 98.96%), followed by RoBERTa (79.66% → 97.82%). These findings demonstrate that class balancing strategy exerts a stronger influence on performance than model architecture alone.

4.3. Unlocked Mobiles Dataset (Oversampling)

This section reports results on the Unlocked Mobiles reviews after class rebalancing via oversampling. As with Fine Food, we present ML and DL results, analyze confusion matrices, and compare models in terms of accuracy, compute, and memory.

Logistic Regression (LR) with TF-IDF features trained on the original (non-rebalanced) training split was used as the global baseline configuration. Baseline performance for both datasets is reported in Appendix A (Table A1). All undersampling and oversampling experiments were evaluated relative to this baseline to ensure controlled and transparent comparison.

4.3.1. ML Results

Table 14. ML Results for Unlocked Mobiles Dataset Using Oversampling.

Algorithm	Accuracy	Precision	Recall	F1	ROC_AUC
LR	0.8930	0.8953	0.8930	0.8934	0.9704
NB	0.8104	0.8106	0.8103	0.8103	0.9358
RF	0.9360	0.9365	0.9359	0.9361	0.9884
GBM	0.6816	0.6789	0.6815	0.6795	0.8504
SVM	0.9362	0.9365	0.9362	0.9361	0.9846

summarizes metrics for Logistic Regression (LR), Naïve Bayes (NB), Random Forest (RF), Gradient Boosting (GBM), and Support Vector Machine (SVM) trained in Google Colab (CPU). Confusion matrices are shown in Figure 12.

SVM and RF are tied at the top on accuracy/F1 (~0.936). RF’s ensemble reduces variance and generalizes well post-oversampling but is compute/memory heavier due to many trees. SVM reaches similar accuracy via a clear margin in the high-dimensional TF-IDF space, typically with lower memory than RF but higher compute than LR. LR remains a strong, efficient baseline (0.8930). NB trails (0.8104), and GBM performs worst (0.6816), consistent with its sensitivity to noisy/redundant duplicated samples created by oversampling.

RF excels on negative/neutral but shows some leakage into positive (e.g., ~1288 misclassifications). LR is balanced across classes, with fewer positive–negative confusions than RF. NB and GBM exhibit higher confusion—especially negative <-> neutral. SVM shows the cleanest diagonals (e.g., 19,757; 19,763; 18,764), making it the best ML choice for this dataset when considering balanced class behavior and accuracy.

Although Random Forest achieved very high performance under oversampling, it is important to note that classical oversampling duplicates minority instances rather than generating new semantic content. Therefore, the observed improvement reflects enhanced class balance rather than structural model innovation. No data leakage occurred, as resampling was strictly applied to the training set only.

4.3.2. DL Results

The deep learning experiments were conducted using Google Colab. CNN and LSTM were trained on CPU, while RoBERTa was fine-tuned using GPU due to its higher computational requirements. Model performance was evaluated across multiple epochs using accuracy, loss, and computational efficiency to assess convergence behavior and generalization.

CNN Performance.

As shown in

Table 15. CNN Results for Unlocked Mobiles Dataset Using Oversampling.

Epoch	Accuracy	Loss	Val Accuracy	Val Loss	Time (s)	Time per Step (ms)
1	0.7349	0.6238	0.8671	0.3713	103	13
2	0.9084	0.2741	0.9172	0.2529	102	13
3	0.9548	0.1440	0.9329	0.2166	102	13
4	0.9719	0.0891	0.9420	0.2142	102	13
5	0.9804	0.0627	0.9465	0.2130	102	13

, CNN was trained for five epochs. Training accuracy increased from 0.7349 (Epoch 1) to 0.9804 (Epoch 5), while validation accuracy improved from 0.8671 to 0.9465. Both training and validation losses decreased steadily (Figure 13), indicating stable convergence without divergence.

The fact that validation accuracy exceeded training accuracy during early epochs suggests effective regularization and good generalization rather than overfitting. Training time remained consistent (≈102 s per epoch; 13 ms per step), reflecting stable computational behavior.

CNN achieved the highest overall validation accuracy (0.9465) among the DL models on the Unlocked Mobiles dataset, demonstrating strong suitability for short, structured review texts.

The LSTM results (

Table 16. LSTM Results for Unlocked Mobiles Dataset Using Oversampling.

Epoch	Accuracy	Loss	Val Accuracy	Val Loss	Time (s)	Time per Step (ms)
1	0.3374	1.0952	0.3461	1.0832	1265	162
2	0.3544	1.0697	0.3913	1.0510	1268	163
3	0.4740	0.9585	0.7826	0.5448	1274	164
4	0.8147	0.4846	0.8434	0.4384	1281	165
5	0.8807	0.3498	0.8776	0.3645	1283	165

) show a slower learning curve, characteristic of sequential architectures. Training accuracy improved from 0.3374 (Epoch 1) to 0.8807 (Epoch 5), while validation accuracy reached 0.8776. Both training and validation losses decreased steadily (Figure 14), indicating progressive learning without instability.

However, computational cost was significantly higher than with CNN (≈1265–1283 s per epoch; 162–165 ms per step) due to recurrent state updates and sequential processing. While LSTM achieved balanced performance, it was less computationally efficient than CNN.

RoBERTa was fine-tuned for three epochs (

Table 17. RoBERTa Results for Unlocked Mobiles Dataset Using Oversampling.

Epoch	Training Loss	Validation Loss	Accuracy	Precision	Recall	F1
1	0.4668	0.3495	0.8795	0.8896	0.8796	0.8811
2	0.1986	0.3290	0.9116	0.9224	0.9117	0.9127
3	0.2932	0.2542	0.9429	0.9459	0.9429	0.9431

). Validation accuracy improved to 0.9429 with an F1-score of 0.9431 at Epoch 3. Validation loss decreased consistently across epochs (Figure 15), confirming stable convergence.

Although training loss slightly increased at Epoch 3, validation loss continued decreasing, suggesting regularization effects rather than instability. Fine-tuning was limited to three epochs because additional training produced marginal gains while increasing overfitting risk.

RoBERTa delivered competitive performance, closely approaching CNN accuracy, while offering stronger contextual modeling capacity.

The aggregated results (

Table 18. DL Results for Unlocked Mobiles Dataset Using Oversampling.

Algorithm	Accuracy	Precision	Recall	F1	ROC_AUC
CNN	0.9465	0.9468	0.9464	0.9462	0.9887
LSTM	0.8776	0.8788	0.8775	0.8778	0.9612
RoBERTa	0.9429	0.9459	0.9429	0.9431	-

) indicate the following:

• CNN achieved the highest accuracy (0.9465)
• RoBERTa followed closely (0.9429)
• LSTM achieved moderate performance (0.8776)

Confusion matrices (Figure 16) show that CNN and RoBERTa produce cleaner class diagonals than LSTM, particularly in separating neutral and positive reviews.

The superior performance of CNN on this dataset can be attributed to the nature of Unlocked Mobiles reviews, which are shorter and lexically focused. Convolutional filters effectively capture localized discriminative n-gram patterns, while transformer-based contextual modeling provides comparatively smaller gains when long-range semantic dependencies are limited.

Oversampling consistently enhanced DL performance by reducing class bias and improving minority-class representation. Among the evaluated architectures:

• CNN provided the best trade-off between accuracy and computational efficiency.
• RoBERTa offered strong contextual modeling with slightly higher computational cost.
• LSTM showed stable learning but incurred substantially higher training time.

These findings confirm that optimal architecture selection is dataset-dependent, and that balancing strategies significantly influence deep model behavior.

As shown in Figure 13b, the steady decline in validation loss confirms stable convergence without divergence, unlike the behavior observed under undersampling.

As shown in Figure 14b, the steady decline in validation loss indicates stable convergence without divergence, contrasting with the behavior observed under undersampling.

Although the training loss shows a slight increase at Epoch 3, the continued decrease in validation loss suggests improved generalization rather than instability. This behavior may be attributed to regularization effects and stochastic optimization dynamics during fine-tuning.

Figure 16 illustrates that CNN and RoBERTa achieve stronger class-wise discrimination than LSTM, as reflected by higher main-diagonal values and fewer inter-class misclassifications, particularly between neutral and positive sentiments.

4.4. Comparison Between Fine Food and Unlocked Mobiles

This section compares ML and DL performance across Fine Food Reviews and Unlocked Mobiles Reviews relative to the Logistic Regression (LR) baseline model and explains why the best-performing model differs by domain [55]. On the Fine Food dataset, there is a clear split between sampling regimes. With undersampling, LR performed best among ML models (accuracy = 0.7414; F1 = 0.7417), while RF lagged (accuracy = 0.6177). Moving to oversampling raised most ML scores—LR improved to accuracy = 0.8250 (F1 = 0.8256), and RF surged to accuracy = 0.9896, indicating strong sensitivity to sample size and class balance. It is important to note that oversampling was applied exclusively to the training split after the train–test partitioning, thereby preventing any information leakage into the validation/test data. SVM was competitive under oversampling (accuracy = 0.8296; F1 = 0.8300).

On Unlocked Mobiles with oversampling, overall results were high, suggesting shorter, more uniform, and more easily separable texts. LR reached accuracy = 0.8930 (F1 = 0.8934) as a strong, computationally efficient baseline. RF and SVM led ML (accuracy = 0.9360 and 0.9362, respectively). NB trailed (accuracy = 0.8104), likely due to its simplistic conditional independence assumption interacting poorly with technical terminology distributions. For DL models, CNN achieved the best accuracy (0.9465), LSTM improved steadily across epochs (to 0.8776), and RoBERTa delivered 0.9429 accuracy (F1 = 0.9431).

These differences align with text nature and length. Fine Food reviews are often longer and more affect-rich, favoring contextual models (e.g., RoBERTa) that capture nuanced sentiment dependencies. In contrast, Mobile reviews tend to be shorter and more technical; pattern-based ML models (e.g., SVM, RF) and CNN excel by leveraging discriminative keywords and local n-gram structures. Length also shapes the feature space: TF-IDF inputs for LR/SVM/RF expand with longer prose (Fine Food), complicating linear or tree boundaries under undersampling, whereas shorter Mobile reviews yield crisper separability. DL models (LSTM/CNN/RoBERTa) benefit when they can model full-sentence context; RoBERTa particularly shines on longer, richer Fine Food texts, while CNN is highly effective on shorter, focused Mobile reviews [55,56].

Importantly, this cross-domain divergence demonstrates that model superiority is not absolute but dataset-dependent [56], which constitutes a key empirical contribution of this study.

4.5. Comparison Between Undersampling and Oversampling

We analyzed the impact of sampling strategies using macro F1-score across models and datasets (

Table 19. Comparison of F1-Scores across Sampling Techniques and Datasets.

Model	Fine Food (Undersampling) F1	Fine Food (Oversampling) F1	Unlocked Mobiles (Oversampling) F1
LR	0.7417	0.8256	0.8934
NB	0.7243	0.7333	0.8103
RF	0.6134	0.9896	0.9361
GBM	0.6567	0.6671	0.6795
SVM	0.7335	0.8300	0.9361
CNN	0.7082	0.9688	0.9465
LSTM	0.7099	0.9547	0.8776
RoBERTa	0.7954	0.9781	0.9431

). The results indicate that oversampling generally improves performance compared to undersampling, particularly for ensemble-based and deep learning models that benefit from larger and more balanced training distributions.

Oversampling produces consistent gains across nearly all models. The most pronounced improvement is observed for Random Forest (RF) on the Fine Food dataset, where the F1-score increases from 0.6134 under undersampling to 0.9896 under oversampling. This substantial gain suggests that RF is highly sensitive to training sample diversity and class balance. With sufficient and balanced data, ensemble trees can model class boundaries more effectively. However, such near-perfect performance should be interpreted cautiously, as synthetic duplication may simplify class separation and inflate performance metrics.

Deep learning models also benefit strongly from oversampling. CNN improves from 0.7082 to 0.9688, LSTM from 0.7099 to 0.9547, and RoBERTa from 0.7954 to 0.9781 on the Fine Food dataset. These results indicate that neural architectures—particularly transformer-based models—require sufficiently balanced and data-rich conditions to capture contextual and semantic nuances effectively.

Classical machine learning models show moderate but consistent improvements. Logistic Regression (LR) increases from 0.7417 to 0.8256, and SVM from 0.7335 to 0.8300 under oversampling. These gains reflect reduced majority-class bias and improved minority-class representation. In contrast, Naïve Bayes (NB) exhibits only marginal improvement (0.7243 to 0.7333), suggesting that its independence assumption limits its responsiveness to distributional balancing.

For the Unlocked Mobiles dataset, performance under oversampling remains strong across models. CNN achieves the highest F1-score among DL methods (0.9465), while SVM and RF lead among ML approaches (0.9361 each). Compared to Fine Food, the performance gap between models is narrower, likely due to the shorter and more structurally uniform nature of Mobile reviews.

These findings confirm that sampling strategy significantly influences model behavior. Oversampling generally enhances performance, especially for ensemble and deep learning models, but its impact varies depending on dataset characteristics such as text length, vocabulary diversity, and class separability. These observations reinforce the importance of aligning model selection and balancing strategy with domain-specific data properties, as discussed in Section 5.

5. Conclusions and Recommendations

5.1. Overall Conclusions

This study presented a systematic and domain-aware investigation of sentiment classification—a supervised subtask of sentiment analysis—under severe class imbalance, using two large-scale e-commerce datasets: Fine Food Reviews and Unlocked Mobile Reviews. While sentiment classification has been widely studied, this work differs from prior studies by explicitly examining how data imbalance, sampling strategies, dataset characteristics, and computational constraints jointly influence model behavior and selection.

Rather than proposing a new algorithm, the novelty of this work lies in its comprehensive, experimentally grounded framework that positions classical machine learning (ML) and deep learning (DL) models as context-dependent solutions, rather than as universally optimal methods. Logistic Regression (LR) and Naïve Bayes (NB) were treated as baseline models, while ensemble methods (RF, GBM), sequence-based DL models (CNN, LSTM), and transformer-based architectures (RoBERTa) were evaluated under identical preprocessing and sampling conditions.

The results demonstrate that data characteristics and balancing strategies are as influential as model architecture. Oversampling consistently improved performance across most models by mitigating majority-class bias, particularly benefiting data-hungry methods such as Random Forest and deep neural networks. However, this improvement came at the cost of increased memory usage and training time, highlighting a critical trade-off between performance and resource efficiency. Although LSTM is no longer considered a state-of-the-art architecture in NLP, it is included as a representative recurrent baseline, providing a meaningful historical and computational reference for understanding how modern transformer models improve contextual representation under class imbalance conditions.

Importantly, the study shows that no single model is universally superior. For longer, linguistically rich reviews (Fine Food dataset), transformer-based contextual modeling (RoBERTa) achieved the highest accuracy and F1-score, confirming its strength in capturing nuanced sentiment expressions. In contrast, for shorter and more technical reviews (Unlocked Mobiles dataset), CNN and margin-based ML models (SVM, RF) performed competitively or even better, due to their ability to exploit localized lexical patterns.

Overall, the findings validate the central argument of this paper: effective sentiment classification under class imbalance requires aligning the model choice with dataset properties, sampling strategy, and computational constraints, rather than relying solely on increasingly complex architectures.

5.2. Practical Implications

The results of this study have several practical implications for researchers and practitioners working on real-world sentiment classification systems. For researchers, it provides a controlled benchmarking framework enabling reproducible comparison between ML and DL models under identical imbalance conditions. For industry practitioners, particularly in e-commerce analytics and customer experience monitoring, the findings provide evidence-based guidance for selecting models according to review length, computational resources, and deployment constraints, consistent with recent NLP studies analyzing customer experience and online review behavior using advanced language models [57,58]. Lightweight models such as LR and SVM offer efficient large-scale deployment options, while transformer-based models provide advantages when deeper contextual understanding is required, as shown in recent work on misinformation detection using pretrained language models [34,58,59].

• Model selection should be domain-driven: Organizations analyzing long-form customer feedback should favor contextual DL models (e.g., RoBERTa), whereas short, structured reviews can be effectively handled by CNN or classical ML methods with significantly lower cost.
• Sampling strategy matters as much as architecture: Oversampling proved to be a strong baseline technique for mitigating class imbalance, particularly when combined with ensemble or DL models.
• Resource-aware deployment is essential: Lightweight models such as LR and SVM remain highly competitive under constrained environments, making them suitable for real-time or large-scale industrial deployment.
• Baseline models should not be overlooked: Simple ML models provide strong reference points and transparency, enabling more meaningful evaluation of advanced DL methods.

These insights support informed decision-making in e-commerce analytics, customer satisfaction monitoring, and opinion mining pipelines [60], where trade-offs between accuracy, interpretability, and computational cost are unavoidable.

5.3. Challenges and Limitations

Despite its contributions, this study has several limitations:

• Data imbalance and noise: While oversampling and undersampling alleviate skewed class distributions, they may introduce duplicated or synthetic artifacts and potential overfitting.
• Label ambiguity: Star-based sentiment labels do not always reflect nuanced emotional content, especially in reviews containing mixed opinions.
• Computational constraints: Training deep models on Google Colab imposed limits on batch size, memory, and the number of epochs, which may have restricted full convergence for some architectures.
• Domain specificity: Both datasets comprise English-language e-commerce reviews, limiting generalizability to other domains or multilingual settings.

These limitations do not invalidate the results but should be considered when interpreting performance comparisons and scalability.

5.4. Future Research Directions

Future work can extend this study in several important directions:

1.

Advanced imbalance handling

Beyond traditional oversampling and undersampling, future studies should explore pseudo-data generation and long-tailed learning approaches [12], such as those proposed in Trusted Multi-view Learning for Long-tailed Classification, to generate more informative minority-class representations.

2.

Aspect-based and fine-grained sentiment analysis

Moving from coarse sentiment classification to aspect-level sentiment modeling would enable more actionable insights for businesses.

3.

Explainable and hybrid models

Integrating explainable AI (XAI) techniques with DL models could improve transparency while retaining high predictive performance.

4.

Cross-domain and multilingual evaluation

Applying the framework to multilingual or cross-domain datasets would further validate its robustness and applicability.

5.

Scalable training environments

Leveraging high-performance computing (HPC) or cloud-based infrastructures would allow deeper architectures, longer training schedules, and more extensive hyperparameter optimization.

5.5. Final Remarks

This work demonstrates that sentiment classification under class imbalance is not solely a modeling challenge but a data-centric and system-level problem. By systematically comparing ML and DL approaches across datasets, sampling strategies, and computational settings, the study bridges the gap between theoretical performance and practical deployment.

Rather than advocating a single “best” model, the paper provides a decision-oriented framework for selecting appropriate techniques based on data characteristics and operational constraints. This contribution strengthens the practical relevance of sentiment analysis research and supports the development of more reliable, scalable, and context-aware opinion mining systems in real-world applications.