A Deep Convolutional Neural Network-Based Model for Aspect and Polarity Classification in Hausa Movie Reviews ^†

Abstract

Aspect-based sentiment analysis (ABSA) plays a pivotal role in understanding the nuances of sentiment expressed in text, particularly in the context of diverse languages and cultures. This paper presents a novel deep convolutional neural network (CNN)-based model tailored for aspect and polarity classification in Hausa movie reviews, as Hausa is an underrepresented language with limited resources and presence in sentiment analysis research. One of the primary implications of this work is the creation of a comprehensive Hausa ABSA dataset, which addresses a significant gap in the availability of resources for sentiment analysis in underrepresented languages. This dataset fosters a more inclusive sentiment analysis landscape and advances research in languages with limited resources. The collected dataset was first preprocessed using Sci-Kit Learn to perform TF-IDF transformation for extracting feature word vector weights. Aspect-level feature ontology words within the analyzed text were derived, and the sentiment of the reviewed texts was manually annotated. The proposed model combines convolutional neural networks (CNNs) with an attention mechanism to aid aspect word prediction. The model utilizes sentences from the corpus and feature words as vector inputs to enhance prediction accuracy. The proposed model leverages the advantages of the convolutional and attention layers to extract contextual information and sentiment polarities from Hausa movie reviews. The performance demonstrates the applicability of such models to underrepresented languages. With 91% accuracy on aspect term extraction and 92% on sentiment polarity classification, the model excels in aspect identification and sentiment analysis, offering insights into specific aspects of interest and their associated sentiments. The proposed model outperformed traditional machine models in both aspect word and polarity prediction. Through the creation of the Hausa ABSA dataset and the development of an effective model, this study makes significant advances in ABSA research. It has wide-ranging implications for the sentiment analysis field in the context of underrepresented languages.

1. Introduction

In today’s digital age, users of movie streaming platforms like Amazon Movies, Netflix, and YouTube have numerous ways to express their opinions through comments and rating systems [1]. These opinions reflect a range of sentiments from positive to negative. Businesses leverage data mining techniques to understand these sentiments, allowing them to improve customer experiences and tailor their offerings to user preferences [2]. However, the challenges arise from factors like misspelled words, abbreviations, emoticons, and slang. As a result, choosing the most effective opinion-mining method is crucial for accurately capturing user sentiment [3].

Sentiment analysis (SA) is a computational method used to identify and categorize opinions expressed in text, aiming to determine whether the writer’s perspective on a particular subject or product is positive, negative, or neutral. Within the broader field of behavioral science, SA is an essential tool for interpreting and classifying emotions across various domains [4]. This area of analysis focuses on understanding people’s sentiments, beliefs, views, and feelings related to products and services, topics, events, and their associated attributes [5].

This study investigates aspect-based sentiment analysis (ABSA) in the context of the Hausa language, which is an area with limited linguistic resources within the field of natural language processing (NLP). The primary objective is to explore the potential of convolutional neural networks (CNNs) as powerful tools for uncovering the nuances of sentiment [6]. Recent research highlights the significance of this endeavor. Despite the challenges posed by scarce linguistic resources, we are motivated to improve the accuracy of sentiment analysis and to shed light on the complex interplay of sentiments across various dimensions. By doing so, we hope to unravel the dynamics of sentiment in this resource-constrained language and deepen our understanding of Hausa’s unique linguistic and cultural context.

While existing studies have examined SA in Hausa, a low-resource language, with a primary focus on SA within an entire document [7,8,9,10,11,12], this research introduces a novel dimension by tapping into ABSA using a generated Hausa YouTube comments dataset. Our approach aims to provide SA at the word level, forging new pathways in this field of inquiry.

There has been swift progress in deep learning and the effectiveness of CNNs in text analysis. CNNs can capture local features, word sequences, and hierarchical structures [13]. Our innovative approach seeks to improve traditional SA by operating at the word level, thus opening novel directions in this research domain.

The rest of the sections are organized as follows: Section 2 examines relevant studies, Section 3 describes the research methodology employed, Section 4 presents the significant results along with a discussion of the findings, and Section 5 concludes the study, offering direction for future research.

Background

ABSA involves the task of discerning the sentiment associated with a particular term within a text. It can be regarded as an advanced form of sentiment analysis, where the key aim is to recognize specific aspect terms within the provided reviews and anticipate the sentiments linked with those distinct aspects. The initial step in conducting ABSA is known as aspect term extraction (ATE), wherein the phrases that represent the core subjects of opinions within a review are identified [14]. To illustrate, consider the review: “I appreciate the scene and the producer’s choice of soundtrack.” In this case, the terms “scene” and “producer” will be extracted as aspect terms, and their sentiment will be determined by the presence of the opinion word “appreciate”. This process constitutes the second subtask of ABSA, which is known as aspect term sentiment classification (ATSC).

The ATE task was considered by some researchers to be a sequential modeling problem where a sentence is modeled as a set of tokens. Various algorithms, ranging from Support Vector Machines (SVMs) [15,16] for ATE to Attention-based LSTM for ATSC [17], have been proposed to facilitate ABSA’s accurate execution.

A considerable range of methods for sentiment analysis across different domains already exists [18,19,20]. The majority of these existing studies mainly concentrate on identifying sentiments at the textual level [21] or in a single sentence [22]. However, these approaches often fall short of providing the specific and detailed insights users seek from reviews, particularly regarding individual aspects, features, or attributes of a product or service. For example, consider a user review “I enjoyed the movie, but the character was annoying”. This demonstrates how viewers can have varied perspectives, where their focus might be on the quality of the movie while at the same time being annoyed by the actor. Hence, it becomes pivotal to gauge sentiments about distinct aspects, which is a task referred to as aspect-based analysis.

ABSA holds the promise of transforming sentiment analysis from a broad perspective to a granular level, offering a more specific understanding of opinions. By analyzing sentiments at the aspect or feature level, ABSA empowers businesses to gain actionable insights from customer feedback [23]. This fine-grained approach not only aids in pinpointing areas of improvement but also enhances market research, competitive analysis, and domain-specific sentiment understanding. ABSA’s potential extends beyond industries; it facilitates personalized recommendations, guides product development, and enables timely responses to evolving sentiment trends. The insights drawn from ABSA support evidence-based decision making in policy and regulations, ensuring that public sentiments are understood and considered, making it an important tool for understanding and harnessing sentiments in diverse contexts.

One of the major challenges of ABSA in low-resource languages is the absence of rich and diverse datasets. Additional challenges include developing a high-quality labeled dataset for training the ABSA model and the scarcity of domain-specific annotated data. Extending ABSA-trained models to different languages, as in the case of multilingual models, also presents difficulties in handling linguistic variations, idiomatic expressions, and sentiment lexicons across diverse languages. Therefore, collecting datasets for each language and training the ABSA model on language-specific datasets is vital to achieving accurate results [24].

ABSA is a specific approach within sentiment analysis that focuses on identifying the sentiment expressed in a sentence or phrase at the aspect level. This is particularly important for low-resource languages like Hausa, as it provide insights into people’s opinions about specific aspects of a product or service, even when the available data is limited.

2. Literature Review

This literature review aims to provide an overview of the current state-of-the-art methods and recent developments in aspect-based sentiment analysis (ABSA) techniques. ABSA evolved naturally as an extension of sentiment analysis to address the need for a more fine-grained analysis of sentiments concerning specific aspects or features. Although many researchers contributed to the development of ABSA, each tried to address a drawback of traditional SA.

Sebastiani et al. [25] conducted a thorough survey of ABSA highlighting challenges and methodologies in aspect-based sentiment analysis. The authors further proposed a technique for identifying aspects and associating sentiments. Tang et al. [26] proposed a technique called the rating regression approach, where aspects are identified and rated by a given score. The mathematical model clearly outlines how ABSA can be further developed. Ever since, more efforts have been made to build an aspect-based analysis system.

Xu et al. [27] introduced SentiWordNet, which is a lexical resource that assigns sentiment scores to words based on their synsets. SentiWordNet has been widely used as a feature in traditional machine learning models for aspect-based sentiment analysis to determine sentiment polarity based on the words associated with aspects. Additionally, Xue et al. [28] demonstrates the use of SentiWordNet in sentiment analysis with Twitter data with a focus on determining sentiment orientation at the document level. This concept was extended by Tang et al. [29] to aspect-level analysis by associating SentiWordNet scores with aspects and utilizing traditional machine learning models to classify the sentiment of individual aspects.

Recently, the popularity of deep learning has led to the integration of neural networks in many fields such as business and healthcare [30]. Also in ABSA, deep learning has brought about substantial advancements. The authors of [31,32,33,34,35], all proposed deep-learning-based ABSA techniques using high-resource language datasets, leading to more accurate and nuanced sentiment analysis [36]. For example, Musa et al. [37] introduced deep learning-based ABSA by proposing an attention-based LSTM model for aspect-level sentiment classification. The authors proposed a hierarchical model that first captures the context of the whole sentence and then focuses on individual aspects using attention mechanisms. The model effectively addresses the challenge of aspect-level sentiment classification by incorporating both local and global context information.

Furthermore, Rosso et al. [38] delved into an end-to-end approach of combining aspect extraction and aspect sentiment classification, which they considered more advantageous than considering the tasks as separate entities. In the past, when working on ABSA, people did not use sentence structure clues well. This made it hard for the aspect extraction to find multi-word aspects, and the aspect sentiment classifier could not understand how words fit together in a sentence. The proposed solutions demonstrate superior performance compared to existing models on the SemEval-2014 dataset, excelling in aspect extraction and aspect sentiment classification subtasks. Movie reviews on discussion boards to explore aspect-based sentiment analysis were proposed [39]. This approach performed sentiment analysis at the clause level to ensure that different aspects were analyzed separately.

Consequently, Essenbar et al. [40] acknowledged the existence of many ABSA works in English using pre-trained models (PTMs) and also highlighted the availability of open datasets. The author stressed that ABSA works in French are few compared to many SA works. This prompted the author to utilize the French Language and three fine-tuning methods such as French PTM multilingual BERT (mBERT), CamemBERT, and FlauBERT on the SemEval2016 datasets. The work surpassed the state-of-the-art French ABSA with flexibility for out-of-domain datasets. other works in low-resource languages, includes a transfer learning-based approach for Arabic ABSA [41,42].

To the best of our knowledge, there have not been many works on aspect-based sentiment analysis in low-resource languages. Therefore, in this work, we tried to make the first attempt at organizing the first Hausa language dataset for ABSA and propose a deep learning model for Hausa aspect-based sentiment analysis.

3. Methodology

3.1. Dataset Description

The dataset is generated from various YouTube channels where comments were made by individuals about Hausa movies. Within this dataset, comments were categorized as either purely Hausa or Engausa (a mixture of Hausa and English). The dataset comprises 590 comments, with four features consisting of comments, aspect words (1—person [actor, actress, director], 2—episode, 3—movie, 4—general), sentiment polarity (negative or positive), and language (Hausa, Engausa).

3.2. Dataset Collection and Preprocessing

Dataset collection was carried out using web scraping with Python 3.7, which facilitated the gathering of numerous Hausa comments from YouTube channels. Subsequently, the dataset underwent preprocessing, involving the removal of instances that did not capture any of the aspect categories and the removal of stopwords—words that typically contribute less meaning to a text, such as ’a,’ ’ni,’ ’to,’ ’su,’an’. Additionally, irrelevant terms, such as emojis, usernames, special characters (@, #), punctuation marks, and numbers, were also removed. Furthermore, the texts were converted to lowercase. Finally, comments and aspect words were vectorized using Term Frequency Inverse Document Frequency (TF-IDF), which is one of the most straightforward and effective ways to comprehend the context of a document and locate relevant information and key terms and phrases.

3.3. Aspect Term Extraction Processing

From the dataset, the aspect terms were manually annotated by native speakers, looking at both the context and semantic meaning of the review. Some predefined rules were followed for aspect term annotation. For example, all proper nouns represent either a person or a movie aspect category. We then applied a supervised learning method for aspect term extraction: specifically, a deep convolutional neural network was used with an attention layer. To enable implicit aspect extraction and understanding, We split the datasets for each task into a training set with 70% of the comments and a test set with the remaining comments. A hyperparameter tuning strategy was utilized to find the best parameter sets.

3.4. Model Building

This section presents the proposed model used in this paper. The model is a deep convolutional neural network with recurrent and attention layers. The recurrent layers allow the model to scan from left to right to identify the semantics of the sentence, while the attention layer allows the model to focus on specific important words in the sentence before the aspect term is extracted. Figure 1 provides a clear description of the entire model framework.

The proposed CNN model architecture can be described as a 9-layer DCNN. The layers are arranged as follows: The first layer is an embedding layer that converts the input data into a dense vector representation. This layer is used to learn the embeddings of the input words, which are then used as input for the rest of the layers. Then, there are two Conv1D layers, which are the convolutional layers of the model. These layers are used to learn feature maps from the input data. The first convolutional layer has 32 filters of size 2, and the second layer has 64 filters of size 2. The next layer is the LSTM layer, which is a recurrent layer that allows the model to pass information from left to right in the sequence. An attention layer is added, which takes embedding from the LSTM layer as input. A GlobalMaxPooling1D layer is added to reduce the spatial dimension of the feature maps learned by the attention layer. The layer is followed by a dense layer with 256 neurons, which is used to learn higher-level features from the feature maps produced by the previous layers. The next layer is a dropout layer, which is used to reduce overfitting by randomly dropping out some of the neurons during training. Finally, a dense layer with 4 units and softmax activation is added as the output layer. The proposed model architecture is presented in Figure 2 to highlight the layers and available parameters.

3.5. Performance Metrics

To assess the performance or effectiveness of a model, system, or process, we employ performance metrics, which are quantitative measures.

• Accuracy is the number of successfully categorized instances (aspect-based sentiment) divided by the total number of cases (aspect-based sentiment instances). It can be calculated using the following formula:

  $$\mathrm{Accuracy}={\displaystyle \frac{\mathrm{True}\mathrm{Positives}+\mathrm{True}\mathrm{Negatives}}{\mathrm{Total}\mathrm{Instances}}}$$
• Precision is calculated as the proportion of true positive instances (positive sentiment polarity) correctly classified as positive. The formula for precision is as follows:

  $$\mathrm{Precision}={\displaystyle \frac{\mathrm{True}\mathrm{Positives}}{\mathrm{True}\mathrm{Positives}+\mathrm{False}\mathrm{Positives}}}$$
• Recall, also known as sensitivity, is the proportion of positive instances correctly classified as positive. The formula for recall is given by

  $$\mathrm{Recall}={\displaystyle \frac{\mathrm{True}\mathrm{Positives}}{\mathrm{True}\mathrm{Positives}+\mathrm{False}\mathrm{Negatives}}}$$
• F1 Score: The F1 score is a metric that combines both precision and recall, striking a balance between them. It can be computed using the following formula:

  $$\mathrm{F}1\mathrm{Score}={\displaystyle \frac{2·\mathrm{Precision}·\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}$$

4. Result and Discussion

4.1. Preliminary Analysis

A descriptive analysis conducted on the dataset revealed that the person aspect term appeared the most, while in terms of sentiment polarity, neutral and positive are the majority classes with a few negative instances. The results of the analysis are presented in Figure 3a, and Figure 3b respectively.

The plots indicate that the aspect-based sentiment from the comments we generated shows that viewers are more interested in the person aspect words, which can be either “director”, “actor”, or “actress”.

4.2. Performance of the Proposed Model

The performance of the proposed deep learning-based aspect-level sentiment analysis is presented in Figure 4 and Figure 5. The model was trained on two separate tasks: identifying aspect words and polarity classification of the aspect words. The model was also evaluated on both aspect word extraction and aspect word polarity classification.

In Figure 3, the performance of the proposed deep learning model is evaluated with accuracy and loss curves. The results indicate superior performance over the baseline machine learning model in identifying aspect words from a given review. The model performance on aspect word polarity classification is presented in Figure 4.

The proposed model achieved remarkable accuracy in aspect word polarity classification of 92%. Compared to baseline models, the proposed model achieved the best results in terms of accuracy in both tasks. This was expected, because a deep neural network with an attention mechanism is the most appropriate for tasks that involve sequential data, which is the case for the extraction of implicit aspect words. Traditional machine learning performs poorly in aspect word detection and polarity classification because the models work better on numerical tabular data. The baseline model results for aspect word detection and polarity classification are presented in

Table 1. Performance of traditional machine learning of aspect word extraction.

Model	Accuracy	Precision	Recall	F1-Score
Naive Bayes	0.70	0.71	0.70	0.67
SVM	0.7153	0.6900	0.7211	0.7021
Random Forest	0.7000	0.72	0.70	0.70
Logistic Regression	0.7624	0.7411	0.7602	0.7511

and

Table 2. Performance of traditional machine learning on aspect polarity.

Model	Accuracy	Precision	Recall	F1-Score
Naive Bayes	0.64	0.60	0.64	0.52
SVM	0.64	0.60	0.64	0.52
Random Forest	0.64	0.60	0.64	0.52
Logistic Regression	0.64	0.60	0.64	0.52

, respectively.

From Table 1 and Table 2, the machine learning methods of logistic regression achieved results superior to the rest of the baseline models in both aspect term detection and classification, while Naive Bayes (NB), Random Forest, and SVM had similar results. The proposed model’s accuracy is compared with the baseline models in

Table 3. Performance comparison between proposed model and machine learning on aspect and polarity tasks.

Model	Accuracy
Results on Aspects Extraction:
Naive Bayes	0.64
SVM	0.64
Random Forest	0.64
Logistic Regression	0.64
Proposed DCNN model	0.91
Results on Polarity Classification:
Naive Bayes	0.64
SVM	0.64
Random Forest	0.64
Logistic Regression	0.64
Proposed DCNN model	0.92

. The results show that the proposed models outperformed the baseline models in terms of accuracy.

5. Conclusions and Future Work

In this research work, we presented a deep learning-based aspect-level sentiment classification of movie reviews in the Hausa language. The main contributions of this work center on providing a Hausa movie review dataset for aspect-level sentiment analysis. We also proposed a CNN model that learns aspect embeddings in Hausa textual data, as Hausa is a language categorized as underrepresented in natural language processing regarding resources. This provides a starting point for aspect-based sentiment analysis in the Hausa language. The proposed model achieves remarkable performance on both aspect term extraction and polarity classification, despite the small size of the dataset. The attention layer allows the model to concentrate on a different part of the sentence to discover aspect terms, allowing it to accurately perform aspect polarity classification. The proposed model outperforms all the compared baseline models which used traditional machine learning models with significant differences.

The major challenge of this work is the inability of the model to analyze more than one aspect word in a sentence at a time. Therefore, an intriguing and potential avenue for future work would be to investigate the potential of transformer architecture in aspect-based sentiment analysis to handle the aforementioned challenge of extracting and analyzing all aspect terms in a sentence simultaneously.