A New Descriptor for Smile Classification Based on Cascade Classifier in Unconstrained Scenarios

Abstract

In the development of human–machine interfaces, facial expression analysis has attracted considerable attention, as it provides a natural and efficient way of communication. Congruence between facial and behavioral inference in face processing is considered a serious challenge that needs to be solved in the near future. Automatic facial expression is a difficult classification issue because of the high interclass variability caused by the significant interdependence of the environmental conditions on the face appearance caused by head pose, scale, and illumination occlusions from their variances. In this paper, an adaptive model for smile classification is suggested that integrates a row-transform-based feature extraction algorithm and a cascade classifier to increase the precision of facial recognition. We suggest a histogram-based cascade smile classification method utilizing different facial features. The candidate feature set was designed based on the first-order histogram probability, and a cascade classifier with a variety of parameters was used at the classification stage. Row transformation is used to exclude any unnecessary coefficients in a vector, thereby enhancing the discriminatory capacity of the extracted features and reducing the sophistication of the calculations. Cascading gives the opportunity to train an extremely precise classification by taking a weighted average of poor learners’ decisions. Through accumulating positive and negative images of a single object, this algorithm can build a complete classifier capable of classifying different smiles in a limited amount of time (near real time) and with a high level of precision (92.2–98.8%) as opposed to other algorithms by large margins (5% compared with traditional neural network and 2% compared with Deep Neural Network based methods).

1. Introduction

Facial expression is one of the potent and prompt means for humans to communicate their emotions, intentions, cognitive states, and opinions to each other [1]. Facial expression plays an important role in the evolution of complex societies, which help to coordinate social interaction, promote group cohesion, and maintain social affiliations [2]. Potential expression recognition technology applications include tutoring systems that are sensitive to students’ expression, computer-assisted deceit detection, clinical disorder diagnosis and monitoring, behavioral and pharmacological treatment assessment, new entertainment system interfaces, smart digital cameras, and social robots. To convey emotions of joy, happiness, and satisfaction, a smile is the most typical facial expression in humans [3]. Modern longitudinal studies have used smile data from images to predict future social and health outcomes [4].

Researchers have made substantial progress in developing automatic facial expression detection systems in the literature [5]. Anger, surprise, disgust, sadness, happiness, and fear are many of the six basic facial expressions and emotions commonly referred to. Among the various facial expressions, happiness, as usually demonstrated in a smile, often occurs in the daily life of a person. Two components of facial muscle movements are included in a smile, namely Cheek Raiser (AU6) and Lip Corner Puller (AU12), as shown in Figure 1 [6]. In computer vision and its fields of operation, the automated facial expression recognition system has become very interesting and difficult [7].

Figure 1. Facial action units (AUs).

In managed settings, current facial expression recognition has promising results, but performance on real-world data sets is still unsatisfactory [8]. This is because there are broad differences in facial appearances through the color of the skin, lighting, posture, expression, orientation, head location, lightening state, and so on. By incorporating deep learning [9], optimization [10], and ensemble classification, automatic methods of identification for facial expression are suggested to deal with existing system difficulties. Five key steps are given for the planned work: preprocessing, a deep evolutionary neural network used for feature extraction, feature selection utilizing swarm optimization, and facial expression classification employing a support vector machine, ensemble classifiers [11], and a neural network [12].

Motivation and Contribution

As an important way to express emotion, facial expression has a vital influence on the communication between people. Currently, in computer vision and pattern recognition, facial expression recognition has become an active research scope. Real-time and effective smile detection can significantly enhance the development of facial expression recognition. The classification of smiles in an unconstrained environment is difficult because of the invertible and wide variety of facial pictures. Faces’ extensive optical alterations, such as occlusions, posture transitions, and drastic lightings, make certain functions very difficult in real-world implementations. The majority of current studies deal with smile detection and not smile classification. However, within the current smile classification approaches, their models are not smile attribute specific hence their performance may be limited.

The main goal of this paper is to build an adaptive model for the classification of smiles that incorporates both row-transformation-based features extraction and a cascade classifier to increase the accuracy of classification. In contrast to the current methods of classifying smiles, which rely on deep neural networks to extract features that, in turn, require a large number of samples and more computation, the suggested model relies on row transformation to reduce and improve the discriminatory capability of the extracted features. Furthermore, the suggested model utilizes the cascade classification concept to build an accurate classifier. Cascading classifiers allow the most likely smile pictures to be evaluated for all features that differentiate an individual. The accuracy of a classifier can even be varied. A chain of experiments proves that the suggested model technique is substantially reliable and quicker than other widespread prototypes.

The remainder of the article is organized as follows. Section 2 discusses the current related work. Section 3 presents the proposed model steps in detail. Section 4 explains experimental designs. Section 5 includes the conclusion and future work.

2. Related Work

Several scientific studies have been performed in the field of identification of facial expressions that apply to a range of technologies such as computer vision, image recognition, bioindustry, forensics, authentication of records, etc. [13,14,15]. In many studies, Principal Component Analysis (PCA) was used to provide a coding framework for facial action that models and recognizes various forms of facial action [16,17]. However, PCA-based solutions are subject to a dilemma in which the projection maximizes variance in all images and negatively affects recognition performance. Independent Component Analysis (ICA) is one of the statistical methods that are adapted to perform expression recognition to elicit statistically independent local face characteristics that proceed better than PCA [18].

Recently, as training and feature extraction are carried out simultaneously, deep learning among the science community has attracted substantial interest in the field of smile detection. The Deep Neural Network (DNN) was the first method of deep study used for the training and classification of models in high-dimensional data [19]. The DNN has one problem: it takes too long to overcome challenges at the preparation stage. The Convolutional Neural Network (CNN) is a deep learning technique that solves DNN problems by reducing preprocessing and thereby enhancing image, audio, and video processing [20,21]. The CNN has great performance while classifying smiles images that are very similar to the data set using a huge computational cost. However, CNNs usually have difficulty in classifying an image that includes some degree of tilt or rotation.

As the feature extraction module represents the core module for facial classification, many algorithms inspired by nature are suggested to select the characteristics of the picture [22], among others, primarily in medicinal applications [23]. In order to choose the optimal features in the face, the feature selection strategy is used to classify the smile of a human by excluding unwanted or redundant features. However, traditional optimization solutions do not maximize and converge to the global minimum solution. Through using metaheuristic evolutionary optimization algorithms such as Ant Colony Optimization (ACO) [24], Bee Colony Optimization (BCO) [25], Particle Swarm Optimization (PSO) [26], etc., conventional techniques will minimize drawbacks. Such approaches are inefficient in evaluating the global optimum concerning the pace of convergence, capability for experimentation, and consistency of solution [27]. An updated Cuckoo Search (CS) algorithm is suggested to take several characteristics to perform classification and uses two learning algorithms, namely K-nearest neighbor and Support Vector Machines (SVMs) [28].

In the literature, many other methods for extracting the salient features of an image have been used, such as the chaotic Gray-Wolf Algorithm [29] and Whale Optimization Algorithm (WOA) [30]. Because randomization is so important in exploration and exploitation, using the existing randomization technique in WOA would raise computational time, especially for highly complex problems. The Multiverse Optimization (MVO) algorithm suffers from a low convergence rate and entrapment in local optima. To overcome these problems, a chaotic MVO algorithm (CMVO) is applied that minimizes the slow convergence problem and traps local optima [31].

A graphical model for the extraction and description of functions using a hybrid approach to recognize a person’s facial expressions was developed in [32]. However, large memory complexity is the main disadvantage. In this case, matrices can also be a good solution when the graph is roughly complete (every node is connected to almost all of the other nodes). In [33], a Zernike model was developed based on a local moment to classify a person’s expressions such as regular, happy, sad, surprise, angry, and fear. Using characteristics for speech recognition and motion change, recognition was done, and SVM was used for the classification. The experiments carried out showed that when compared to the individual descriptor, the integrated system achieves better results. However, this takes a long training time and has a large difficulty to understand and interpret the final model, variable weights, and individual impact.

Recently, several methods for classifying face speech using a neural network approach have been suggested [34,35]. A target-oriented approach using a neural network for facial expression detection was discussed in [34]. There are many limitations of this approach such as stated goals may not be realistic, and unintended outcomes may be ignored. In [36], the detection technique was used to perform automatic recognition of facial expressions using the Elman neural network to recognize feelings such as satisfaction, sadness, frustration, anxiety, disgust, and surprise. The identification rate was analyzed to be lower for pictures of sorrow, anxiety, and disgust. However, neural networks demand processors with parallel processing power by their structure. Furthermore, experience and trial and error are used to achieve the appropriate network structure.

Inspired by the good performance of the CNNs in computer vision tasks, such as image classification and face recognition, several CNN-based smile classification approaches have been proposed in recent years. In [37], a deeper CNN that has a complex CNN network consisting of two convolution layers, each accompanied by a max-pooling and four initiation layers, was suggested for facial expression recognition. It has a single-part architecture that takes face pictures as input and classifies them into one of the seven sentences. Another related work in [38] utilizes deep learning-based facial expression to minimize the dependency on face physics. Herein, the input image is convoluted in the convolution layers with a filter set. To identify the facial expression, the CNN generates a map of functions that are then paired with fully connected networks. In [39], a deep learning approach is introduced to track consumer behavior patterns by measuring customer behavior patterns. The authors in [40] presented a deep region and multilabel learner’s scheme for estimation of head poses and study of facial expressions to report the interest of customers. They used a feedback network to isolate vast facial regions.

In general, a deep learning approach gives optimum facial features and classification. However, it is difficult to gather vast amounts of training data for facial expression recognition under different circumstances and more massive calculations are required. Therefore, the calculation time of the deep learning algorithm needs to be reduced. In order to minimize the number of features, a Deep Convolutional Neural Network (DCNN) and Cat Swarm Optimization (CSO) are used for facial expression recognition methods to minimize processing time [10]. Yet, there is no common theory to help choose the best resources for deep learning because they need an understanding of the topology, the process of training, and other parameters; as a consequence, fewer experienced individuals find it impossible to follow.

In contrast to the previous methods, which rely on a deep learning concept for smile classification, and in order to solve the problem facing this type of learning in terms of its difficulty to gather vast amounts of training data for facial expression recognition under different circumstances, the suggested approach utilizes both the row transformation technique and the cascade classifier in a unified framework. The cascade classifier can process a large number of features. Even so, the effectiveness of this method is fundamentally dependent on the extracted features, which may indeed not require much time to realize its purpose algorithm. In this case, row transformation is used to exclude any redundant coefficients from a vector of features, thus increasing the discriminatory capacity of the derived features and reducing computational complexity.

3. Methodology

3.1. Face Detection

The first step in the identification of a smile is to locate the face in the picture. For this function, the Viola–Jones method was used [41]. The face identified represents a Region of Interest (ROI) in the picture of a smile. The Viola–Jones method was also used to locate the eyes and mouth. The area of the eyebrow was determined from the position of the eye region. After the identification of facial regions, different techniques of image processing were used in each of the detected ROIs to remove the eyes and mouth. A search was then carried out on each of the extracted components to identify facial expressions [42]. Figure 2 shows the proposed system block diagram, and Figure 3 shows an example of the facial regions.

Figure 2. System flowchart.

Figure 3. Facial regions. (a) Input, (b) face detection (Step 1), and (c) region detection (Step 2).

To soften the image, minor noises such as defects in the image and scarcely visible lines were discarded. In order to locate points of interest on the face, it was initially important to enhance and extract the relevant information from the image. For this reason, different techniques of image processing were used in this work such as contrast correction, thresholding, context subtraction, contour detection, and Laplacian and Gaussian filters for the extraction of points of interest. To segment the image into regions (set of pixels), two methods were used in segmentation: thresholding and morphological operations. To remove the edges of the eyes and mouth, a canny detector was used, and a search was carried out on each of the resulting edges to detect facial landmarks. Figure 4 illustrates the output facial landmarks, which are detected from the image processing techniques in each of the ROIs (see [42] for in-depth details).

Figure 4. Facial landmarks.

3.2. Feature Extraction

After the preprocessing stage, feature extraction is performed in a facial expression recognition system. The most important knowledge present in the original ROI is a kind of dimensional decrease technique. This is the knowledge gathered in a small space from the photo. The main goal of the extraction function is to minimize the initial ROI size into a manageable processing vector that has histogram and alpha and beta features.

3.2.1. Histogram Feature Extraction

Herein, six parameters of histograms are calculated for each ROI. The histogram features are statically based features as a model of the probability distribution of the gray levels. We define the first-order histogram probability as [43]:

$$P\left(g\right)=\frac{N\left(g\right)}{M}$$

(1)

$N\left(g\right)$ is the number of pixels at gray level value g, and M is the number of pixels in the ROI. $P\left(g\right)$ has all values less than or equal to 1. The total number of gray levels available will be L, so the gray levels range from 0 to L – 1. Histogram probabilities include the mean, standard deviation, skew, Kurtosis, energy, and entropy. Mean $\left(\mu \right)$ is the average value, which informs us more about the ROI’s overall brightness. The mean of a light ROI will be high, while the mean of a dark ROI will be low. Standard deviation ($\sigma )$ shows contrast and represents the distribution of data; a high-contrast image has a high variance, while a low-contrast image has a low variance. Skewness ($P_3$) is a metric for symmetry or, more specifically, its absence. A distribution, or data collection, is said to be symmetrical if it appears identical on both sides of the middle point. Kurtosis ($P_4$) is a statistic that indicates when data are heavily or lightly skewed in comparison to a standard distribution. Energy $\left(\zeta \right)$ reveals knowledge about how the gray levels are distributed. For an image with a constant value, the measure of energy has a maximal value of 1 and gets increasingly smaller as the pixel values are distributed across grayer level values. The greater this value, the more easily the ROI data can be compressed. If the energy is high, it indicates that the ROI has a limited number of gray levels, implying that the distribution is concentrated in just a few different gray levels. Entropy $\left(\eta \right)$ is a measure that informs the number of bits required to code the ROI data. The entropy of the ROI grows as the pixel values are spread over more gray depths. This value is usually inversely proportional to the energy [43].

$$\mu ={\displaystyle \sum }_{g=0}^{L-1} g P\left(g\right)$$

(2)

$$\sigma ={\displaystyle \sum }_{g=0}^{L-1} {\left(g-P_1\right)}^2 P\left(g\right)$$

(3)

$$P_z={\displaystyle \sum }_{g=0}^{L-1} \frac{{\left(g-\mu \right)}^z P\left(g\right)}{{({\mu }_1)}^{\raisebox{1ex}{$z$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}} z\in \left\{3,4\right\}$$

(4)

$$\zeta ={\displaystyle \sum }_{g=0}^{L-1} {\left[P\left(g\right)\right]}^2$$

(5)

$$\eta =-{\displaystyle \sum }_{g=0}^{L-1} P\left(g\right){\log }_2\left[P\left(g\right)\right]$$

(6)

3.2.2. Alpha and Beta Features

Alpha and beta are the comparisons between the area of teeth and lips. In order to reduce the amount of redundant information, the oral region needs to be extracted, and the lip area, teeth area, and eye area are taken as regions of interest. A method based on a localized active contour model can segment the mouth area by general structure and face proportion (see [44] for all method details). Figure 5 illustrates the steps to pick the lips area.

$$Alpha \left(\alpha \right)=teeth\_Area/Lips\_Area$$

(7)

$$Beta\left(\beta \right)=Lip\_Width/Eye\_Length$$

(8)

Figure 5. For each smile sample in (1), (2), and (3) (a) From left to right: RGB images after brightness equalization, outside contours, internal contours, outcomes of the outside boundary, outcomes of the internal boundary, and finishing segmentation outcomes; (b) convergence outcomes of the outside and internal contours; (c) segmentation outcomes of images in (b).

3.3. Cascade Classifier

The cascade classifier consists of several strong classifiers. The classifiers in the earlier stages are simple and can speedily filter out the background regions. They are more complex in the classifiers of the later stages to spend more computational time on the promising face-like regions. In our case, the features are combined to be used in the MATLAB toolbox. The proposed system is employed with different classifiers such as AdaBoostM2, RUSBoost, and Bagging. AdaBoost is flexible, so the learners may be fine-tuned after being misclassified in the prior classifications of the initial learner. Such issue types might be more prone to overfitting than others. As long as each individual’s output has a higher than random chance of creating improved results, the final model may be considered a convergent learner. Although improving accuracy can improve the learner’s score, it sacrifices understanding and simplification. Additionally, it can be difficult to execute because of the increased computing requirements. Herein, these classifiers were used as a black box with their default parameters (see [11] for more details). Figure 6 illustrates some different smile categories.

Figure 6. Different smiles for a person.

4. Experimental Results

The proposed facial expression recognition system is tested with a data set of benchmark data sets that includes the Japanese Female Facial Expression (JAFFE), Extended Cohn–Kanade (CK+), and CK+48 data sets [43,44,45]. JAFFE is a Japanese database containing 7 facial expressions with a 256 × 256-pixel resolution of 213 images. With 10,414 images with a resolution of 640 × 490 pixels, the CK+ database has 13 expressions. The CK+48 data set has 7 facial expressions with a resolution of 48 × 48 pixels with 981 images. Features are extracted from ROIs using histograms and lip, teeth, and eye areas, which produce a 21-dimensional feature vector. Herein, 80 percent is selected for training, and 20 percent is for the testing of each data set considered. The prototype classification methodology was developed in a modular manner and implemented and evaluated on a Dell^TM Inspiron^TM N5110 laptop device, manufactured by Dell Computer Corporation in Round Rock, Texas, U.S. with specifications Intel(R) Core(TM) i5-2410M processor running at 2.30 GHz, 4.00 GB of RAM, Windows 7 64-bit. Herein, recognition rate, accuracy, sensitivity, recall, specificity, precision,$F_{measure}$, and sensitivity are used to evaluate the efficiency of the suggested model. See [39] for more details.

$$Rate=\frac{No. of expressions classified correctly }{Total no. of images}\times 100$$

(9)

$$Precision=Positive Predictive Value \left(PPV\right)=\frac{TP}{TP+FP}\times 100$$

(10)

$$Sensitivity=Recall=Hit rate=True Positive Rate \left(TPR\right)=\frac{TP}{TP+FN}\times 100$$

(11)

$$F_{measure}=2\times \frac{PPV\times TPR}{PPV+TPR}\times 100$$

(12)

$$Specificity=Selectivity=True Negative Rate \left(TNR\right)=\frac{TN}{TN+FP}$$

(13)

$$Accuracy=\frac{TP+TN}{TP+TN+FN+FP}\times 100$$

(14)

where TP, TN, FP, and FN are the true positive, true negative, false positive, and false negative, respectively. Herein, 80% of samples for each class were used for training, and the remaining 20% of samples were used for testing.

Performance Analysis

The first set of experiments was conducted to verify the efficiency of the suggested model under different cascade classifiers for different data sets that represent different conditions such as occlusions, pose changes, and extreme lightings. Table 1, Table 2 and Table 3 illustrate the statistical results of applying the suggested smile classification model using Adaboost, Bagging, and RUSBoost classifiers, respectively. In our evaluation using standard benchmarks, the suggested model achieved accuracy over 98%, 91%, and 65% for CK+ and CK+48 and JAFFE. respectively. On average, the suggested model requires about 1 s to execute loading, preprocessing, and feature building for each image of size 256 × 256. The cascade classification process requires about 13 s for training and to perform the classification of 2000 images. This runtime depends on the capabilities of the device used, and this runtime can be reduced by using a device that has higher specifications.

Table 1. Statistical analysis of various data sets using the AdaboostM1 classifier.

Data Set	TP	TN	FP	FN	Accuracy	TPR	TNR	PPV	${\mathit{F}}_{\mathit{m}\mathit{e}\mathit{a}\mathit{s}\mathit{u}\mathit{r}\mathit{e}}$
CK+	2048.9	1916.59	33.1	33.1	98.35	0.98	0.98	0.98	0.98
CK+48	180.2	165.74	15.8	15.8	91.64	0.92	0.92	0.92	0.92
JAFFE	26.5	34.22	16.5	16.5	65.02	0.62	0.68	0.62	0.62

Table 2. Statistical analysis of various data sets using the Bagging classifier.

Data Set	TP	TN	FP	FN	Accuracy	TPR	TNR	PPV	${\mathit{F}}_{\mathit{m}\mathit{e}\mathit{a}\mathit{s}\mathit{u}\mathit{r}\mathit{e}}$
CK+	2049.2	1919.32	32.8	32.8	98.38	0.98	0.98	0.98	0.98
CK+48	179.8	165.66	16.2	16.2	91.44	0.92	0.91	0.92	0.98
JAFFE	27.3	34.47	15.7	15.7	66.62	0.65	0.69	0.63	0.64

Table 3. Statistical analysis of various data sets using the RUSBoost classifier.

Data Set	TP	TN	FP	FN	Accuracy	TPR	TNR	PPV	${\mathit{F}}_{\mathit{m}\mathit{e}\mathit{a}\mathit{s}\mathit{u}\mathit{r}\mathit{e}}$
CK+	2047.5	1919.19	34.5	34.5	98.29	0.98	0.98	0.98	0.98
CK+48	179.7	165.67	16.3	16.3	91.39	0.92	0.91	0.92	0.92
JAFFE	26.2	34.46	16.8	16.8	64.64	0.61	0.68	0.61	0.61

It is noted that the proposed system does not achieve good results in the case of the JAFFE data set, as this benchmark database includes an insufficient number of images for each class. In general, utilizing cascading classifiers need more data for correct training. Results in the tables reveal that there are no clear differences between the uses of the different cascade classifiers for the proposed model of the same database in terms of different objective measurements. The results confirm the research hypothesis that using cascade classifiers based on discriminative features will enhance the classification accuracy. The previous results were confirmed through confusion matrix analysis for the Bagging classifiers using the three benchmark data sets, as illustrated in Table 4, Table 5 and Table 6. During the training stage, Bagging and Boosting get N learners by generating additional data. N new training data sets are produced by random sampling with substitution from the original set. Some observations may be replicated in each new training data set by sampling with replacement. In Bagging, any variable has the same potential of appearing in a new data set. To expand on this idea, though, some of the observations are updated or supplemented as often as Boosting requires. This learning algorithm will be trained using multiple sets of multiple samples to avoid issues that could arise due to multiple classifiers training on the same data [45].

Table 4. Confusion matrix using the Bagging classifier for the CK+48 data set (20% of samples for testing).

Emotions	E1	E2	E3	E4	E5	E6	E7
E1	31	0	0	0	0	0	0
E2	2	5	0	0	0	0	0
E3	0	5	24	0	0	0	0
E4	0	0	5	16	0	0	0
E5	0	0	0	1	35	0	0
E6	0	0	0	0	3	13	0
E7	0	0	0	0	0	4	52

Table 5. Confusion matrix using the Bagging classifier for the JAFFE data set (20% of samples for testing).

Emotions	E1	E2	E3	E4	E5	E6	E7
E1	15	0	0	0	0	0	0
E2	3	3	0	0	0	0	0
E3	0	3	3	0	0	0	0
E4	0	0	2	2	0	0	0
E5	0	0	0	1	1	0	0
E6	0	0	0	0	1	6	0
E7	0	0	0	0	0	1	2

Table 6. Confusion matrix using the Bagging classifier for the CK+ data set (20% of samples for testing).

Emotions	E1	E2	E3	E4	E5	E6	E7	E8	E9	E10	E11	E12	E13
E1	418	0	0	0	0	0	0	0	0	0	0	0	0
E2	4	371	0	0	0	0	0	0	0	0	0	0	0
E3	0	3	341	0	0	0	0	0	0	0	0	0	0
E4	0	0	3	329	0	0	0	0	0	0	0	0	0
E5	0	0	0	4	231	0	0	0	0	0	0	0	0
E6	0	0	0	0	3	209	0	0	0	0	0	0	0
E7	0	0	0	0	0	5	85	0	0	0	0	0	0
E8	0	0	0	0	0	0	5	25	0	0	0	0	0
E9	0	0	0	0	0	0	0	5	3	0	0	0	0
E10	0	0	0	0	0	0	0	0	2	8	0	0	0
E11	0	0	0	0	0	0	0	0	0	2	14	0	0
E12	0	0	0	0	0	0	0	0	0	0	2	5	0
E13	0	0	0	0	0	0	0	0	0	0	0	2	3

The second set of experiments demonstrated how the classification rate of the proposed model is dependent on the number of facial image samples enrolled in the smile’s class, so the chance of a correct hit rises as the smile’s class enrolls more samples. In Table 7, Table 8 and Table 9 for each cascade classifier, as expected, the classification rate goes up in line with the interclass term variation. The accuracy rate increases by about 4% on average with every 100 samples added to the data set. By combining both samples to develop the proposed model, the precision is increased (on average) by 98.29%, 91.64%, and 65.02% for CK+, CK+48, and JAFFE data sets. respectively.

Table 7. Relationship between accuracy rate and the number of samples using the AdaboostM1 classifier.

Data Set	Samples Numbers	Accuracy Rate
CK+	1000	80.01
	2000	82.10
	3000	85.00
	4000	87.00
	5000	88.02
	6000	89.02
	7000	91.01
	8000	95.00
	9000	97.05
	10,414	98.35
CK+48	100	70.01
	200	75.06
	300	76.08
	400	79.80
	500	81.02
	600	81.06
	700	85.00
	800	89.90
	981	91.64
JAFFE	40	48.00
	80	49.00
	120	60.03
	160	62.50
	213	65.02

Table 8. Relationship between accuracy rate and the number of samples using the Bagging classifier.

Data Set	Samples Numbers	Accuracy Rate
CK+	1000	81.81
	2000	83.15
	3000	84.50
	4000	86.00
	5000	85.02
	6000	88.02
	7000	90.01
	8000	92.00
	9000	96.05
	10,414	98.38
CK+48	100	71.01
	200	73.05
	300	75.28
	400	77.85
	500	80.52
	600	82.46
	700	85.50
	800	88.95
	981	91.44
JAFFE	40	42.00
	80	44.00
	120	55.03
	160	60.50
	213	66.62

Table 9. Relationship between accuracy rate and the number of samples using the RUSBoost classifier.

Data Set	Samples Numbers	Accuracy Rate
CK+	1000	82.82
	2000	84.35
	3000	84.58
	4000	85.55
	5000	86.08
	6000	88.72
	7000	91.51
	8000	92.07
	9000	97.07
	10,414	98.29
CK+48	100	71.61
	200	72.65
	300	75.26
	400	76.85
	500	82.72
	600	84.47
	700	85.57
	800	89.75
	981	91.39
JAFFE	40	47
	80	55.05
	120	60.03
	160	62.5
	213	64.64

The third set of experiments was conducted to verify how the proposed model’s verification rate varies with the amount of noise in the picture. In this scenario, the image is reinforced with Gaussian noise (noise amount between 1 and 10). As seen in Table 10, the probability of a correct hit decreases with increasing noise. Up to a degree of noise, however, output gains decline with the noise levels in the class’s image. When noise alters the gray level of the images, a variation in the derived features occurs. In these three data sets, we observe the same disparity in precision.

Table 10. Relationship between accuracy rate and noise amount.

Data Set	Noise Amount	Accuracy Rate
CK+	1	98.01
	2	96.5
	5	95.02
	7	93.54
	10	92.68
CK+48	1	90.72
	2	89.23
	5	87.25
	7	85.03
	10	84.38
JAFFE	1	65.73
	2	62.32
	5	61.96
	7	61.61
	10	60.63

The final series of experiments validated the proposed model’s efficiency in comparison to the state-of-the-art models mentioned in Table 11 using the CK+ data set. The findings corroborate the proposed model’s dominance. Despite the proposed model’s convergence with the 3D Shape-based recognition model’s performance, the suggested model is descriptor-independent (geometric descriptor), and it employs a number of translation- and scale-invariant functions. By and large, the 3D Shape descriptor performs poorly, while the data collection contains more noise, i.e., target groups overlap. Furthermore, employing a Deep neural network needs adjusting network configuration parameters that, in turn, need more effort.

Table 11. Comparisons with different methods on CK+.

Method	Accuracy Rate
Neural Network [36]	94.4
Deep Neural Network [37]	97.8
3D Shape [46]	86.8
Gabor [47]	91.81
LBP [47]	82.38
MSDF [47]	94.34
Simple BoW [47]	92.67
SS-SIFT+BoW [47]	93.28
MSDF+BoW [47]	95.85
3D Shape + GA [48]	97.6
3D Shape + GA + KSS [48]	97.9
Gabor [49]	93.8
2D Shape [50]	92.4
The proposed work	98.01

5. Conclusions

Facial expression classification is a very challenging and open area of research. This paper developed a simple yet effective smile classification approach based on a combination of a row-transform-based feature extraction algorithm and a cascade classifier. Utilizing row transformation helps to remove some unnecessary coefficients from the extracted features’ vector to reduce computational complexity. By taking a weighted average of the decisions made by poor learners, cascading assists in training a highly reliable classifier. The model’s objective is to achieve the lowest possible recognition error, the shortest possible run time, and the simplest layout. For various samples, the model achieves a strong identification accuracy of 98.69%. The proposed model is characterized by simplicity in implementation, in contrast to deep-learning-based classification methods that depend on adjusting multiple variables to achieve reliable accuracy. On the other hand, the limitation of this work appeared in the JAFFE data set because of the insufficient number of samples. In the future, a mobile application shall be created to find expressions in each video frame automatically. Furthermore, speech detection includes both audios from a speaker tone, and video responses can further improve detection accuracy.