Robust Facial Expression Recognition Using an Evolutionary Algorithm with a Deep Learning Model
Abstract
:1. Introduction
2. Literature Review
3. The Proposed Model
3.1. Histogram Equalization
- The kth intensity level in the [0, L1] range is represented by the value Xk.
- If nk is large, the input image has a large number of pixels.
3.2. Feature Extraction
3.3. Hyperparameter Tuning
3.4. Facial Expression Classification
Algorithm 1: Teaching stage |
Do then End if for |
Algorithm 2: Learning stage |
then Else then End if End for |
4. Results and Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Fathima, A.; Vaidehi, K. Review on facial expression recognition system using machine learning techniques. In Advances in Decision Sciences, Image Processing, Security and Computer Vision; Springer: Cham, Switzerland, 2020; pp. 608–618. [Google Scholar]
- Ravichandran, T. An Efficient Resource Selection and Binding Model for Job Scheduling in Grid. Eur. J. Sci. Res. 2012, 81, 450–458. [Google Scholar]
- Revina, I.M.; Emmanuel, W.S. A survey on human face expression recognition techniques. J. King Saud Univ. Comput. Inf. Sci. 2021, 33, 619–628. [Google Scholar] [CrossRef]
- Mohan, P.; Thangavel, R. Resource Selection in Grid Environment based on Trust Evaluation using Feedback and Performance. Am. J. Appl. Sci. 2013, 10, 924–930. [Google Scholar] [CrossRef] [Green Version]
- Jaswanth, K.S.; David, D.S. A novel based 3D facial expression detection using recurrent neural network. In Proceedings of the 2020 International Conference on System, Computation, Automation and Networking (ICSCAN), Pondicherry, India, 3–4 July 2020; pp. 1–6. [Google Scholar]
- Jain, D.K.; Zhang, Z.; Huang, K. Multi angle optimal pattern-based deep learning for automatic facial expression recognition. Pattern Recognit. Lett. 2017, 139, 157–165. [Google Scholar] [CrossRef]
- Vo, T.-H.; Lee, G.-S.; Yang, H.-J.; Kim, S.-H. Pyramid With Super Resolution for In-the-Wild Facial Expression Recognition. IEEE Access 2020, 8, 131988–132001. [Google Scholar] [CrossRef]
- Zheng, H.; Wang, R.; Ji, W.; Zong, M.; Wong, W.K.; Lai, Z.; Lv, H. Discriminative deep multi-task learning for facial expression recognition. Inf. Sci. 2020, 533, 60–71. [Google Scholar] [CrossRef]
- Hardas, B.M.; Pokle, S.B. Optimization of Peak to Average Power Reduction in OFDM. J. Commun. Technol. Electron. 2017, 62, 1388–1395. [Google Scholar] [CrossRef]
- Hardas, B.M.; Pokle, S.B. Analysis of OFDM system using DCT-PTS-SLM based approach for multimedia applications. Clust. Comput. 2018, 22, 4561–4569. [Google Scholar] [CrossRef]
- Rajan, S.; Chenniappan, P.; Devaraj, S.; Madian, N. Novel deep learning model for facial expression recognition based on maximum boosted CNN and LSTM. IET Image Process. 2020, 14, 1373–1381. [Google Scholar] [CrossRef]
- Sikkandar, H.; Thiyagarajan, R. Deep learning based facial expression recognition using improved Cat Swarm Optimization. J. Ambient. Intell. Humaniz. Comput. 2020, 12, 3037–3053. [Google Scholar] [CrossRef]
- Wang, K.; Peng, X.; Yang, J.; Lu, S.; Qiao, Y. Suppressing uncertainties for large-scale facial expression recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2020, Seattle, WA, USA, 14–19 June 2020; pp. 6897–6906. [Google Scholar]
- Li, J.; Jin, K.; Zhou, D.; Kubota, N.; Ju, Z. Attention mechanism-based CNN for facial expression recognition. Neurocomputing 2020, 411, 340–350. [Google Scholar] [CrossRef]
- Cheng, S.; Zhou, G. Facial Expression Recognition Method Based on Improved VGG Convolutional Neural Network. Int. J. Pattern Recognit. Artif. Intell. 2019, 34, 2056003. [Google Scholar] [CrossRef]
- Cao, S.; Yao, Y.; An, G. E2-capsule neural networks for facial expression recognition using AU-aware attention. IET Image Process. 2020, 14, 2417–2424. [Google Scholar]
- Kim, J.-H.; Kima, B.-G.; Roy, P.P.; Jeong, D.-M. Efficient Facial Expression Recognition Algorithm Based on Hierarchical Deep Neural Network Structure. IEEE Access 2019, 7, 41273–41285. [Google Scholar] [CrossRef]
- Zhu, Q.; Mao, Q.; Jia, H.; Noi, O.E.N.; Tu, J. Convolutional relation network for facial expression recognition in the wild with few-shot learning. Expert Syst. Appl. 2021, 189, 116046. [Google Scholar] [CrossRef]
- Liu, S.; Gao, P.; Li, Y.; Fu, W.; Ding, W. Multi-modal fusion network with complementarity and importance for emotion recognition. Inf. Sci. 2023, 619, 679–694. [Google Scholar] [CrossRef]
- Liu, S.; Huang, S.; Fu, W.; Lin, J.C.-W. A descriptive human visual cognitive strategy using graph neural network for facial expression recognition. Int. J. Mach. Learn. Cybern. 2022, 1–17. [Google Scholar] [CrossRef]
- Mahmood, M.R.; Abdulrazaq, M.B.; Zeebaree, S.R.M.; Ibrahim, A.K.; Zebari, R.R.; Dino, H.I. Classification techniques’ performance evaluation for facial expression recognition. Indones. J. Electr. Eng. Comput. Sci. 2020, 21, 1176–1184. [Google Scholar] [CrossRef]
- Abdulrazaq, M.B.; Mahmood, M.R.; Zeebaree, S.R.M.; Abdulwahab, M.H.; Zebari, R.R.; Sallow, A.B. An Analytical Appraisal for Supervised Classifiers’ Performance on Facial Expression Recognition Based on Relief-F Feature Selection. J. Phys. Conf. Ser. 2021, 1804, 012055. [Google Scholar] [CrossRef]
- Rajaraman, P.V.; Prakash, M. Intelligent deep learning based bidirectional long short term memory model for automated reply of e-mail client prototype. Pattern Recognit. Lett. 2021, 152, 340–347. [Google Scholar] [CrossRef]
- Wu, M.; Su, W.; Chen, L.; Pedrycz, W.; Hirota, K. Two-Stage Fuzzy Fusion Based-Convolution Neural Network for Dynamic Emotion Recognition. IEEE Trans. Affect. Comput. 2020, 13, 805–817. [Google Scholar] [CrossRef]
- Neelakandan, S.; Paulraj, D.; Ezhumalai, P.; Prakash, M. A Deep Learning Modified Neural Network(DLMNN) based proficient sentiment analysis technique on Twitter data. J. Exp. Theor. Artif. Intell. 2022, 1–20. [Google Scholar] [CrossRef]
- Sunitha, G.; Geetha, K.; Neelakandan, S.; Pundir, A.K.S.; Hemalatha, S.; Kumar, V. Intelligent deep learning based ethnicity recognition and classification using facial images. Image Vis. Comput. 2022, 121, 104404. [Google Scholar] [CrossRef]
- Neelakandan, S.; Arun, A.; Bhukya, R.R.; Hardas, B.M.; Kumar, T.C.A.; Ashok, M. An Automated Word Embedding with Parameter Tuned Model for Web Crawling. Intell. Autom. Soft Comput. 2022, 32, 1617–1632. [Google Scholar] [CrossRef]
- Kavitha, M.; Babu, B.S.; Sumathy, B.; Jackulin, T.; Ramkumar, N.; Manimaran, A.; Walia, R.; Neelakandan, S. Convolutional Neural Networks Based Video Reconstruction and Computation in Digital Twins. Intell. Autom. Soft Comput. 2022, 34, 1571–1586. [Google Scholar] [CrossRef]
- Chandrasekaran, S.; Singh Pundir, A.K.; Lingaiah, T.B. Deep Learning Approaches for Cyberbullying Detection and Classification on Social Media. Comput. Intell. Neurosci. 2022, 2022, 2163458. [Google Scholar] [CrossRef]
- Bulut, F. Low dynamic range histogram equalization (LDR-HE) via quantized Haar wavelet transform. Vis. Comput. 2021, 38, 2239–2255. [Google Scholar] [CrossRef]
- Rahman, M.T.; Dola, A. Automated Grading of Diabetic Retinopathy using DenseNet-169 Architecture. In Proceedings of the 2021 5th International Conference on Electrical Information and Communication Technology (EICT), Khulna, Bangladesh, 17–19 December 2021; pp. 1–4. [Google Scholar]
- Jia, H.; Sun, K.; Zhang, W.; Leng, X. An enhanced chimp optimization algorithm for continuous optimization domains. Complex Intell. Syst. 2021, 8, 65–82. [Google Scholar] [CrossRef]
- Lei, J.; Liu, C.; Jiang, D. Fault diagnosis of wind turbine based on Long Short-term memory networks. Renew. Energy 2019, 133, 422–432. [Google Scholar] [CrossRef]
- Ashtiani, M.N.; Toopshekan, A.; Astaraei, F.R.; Yousefi, H.; Maleki, A. Techno-economic analysis of a grid-connected PV/battery system using the teaching-learning-based optimization algorithm. Sol. Energy 2020, 203, 69–82. [Google Scholar] [CrossRef]
- Available online: https://www.kaggle.com/datasets/shawon10/ckplus (accessed on 14 October 2022).
- Saeed, S.; Shah, A.A.; Ehsan, M.K.; Amirzada, M.R.; Mahmood, A.; Mezgebo, T. Automated Facial Expression Recognition Framework Using Deep Learning. J. Healthc. Eng. 2022, 2022, 5707930. [Google Scholar] [CrossRef] [PubMed]
- Farah Sayeed, R.; Princey, S.; Priyanka, S. Deployment of Multicloud Environment with Avoidance of DDOS Attack and Secured Data Privacy. Int. J. Appl. Eng. Res. 2015, 10, 8121–8124. [Google Scholar]
- Awari, H.; Subramani, N.; Janagaraj, A.; Thanammal, G.B.; Thangarasu, J.; Kohar, R. Three-dimensional dental image segmentation and classification using deep learning with tunicate swarm algorithm. Expert Syst. 2022, e13198. [Google Scholar] [CrossRef]
- Subbulakshmi, P.; Prakash, M.; Ramalakshmi, V. Honest Auction Based Spectrum Assignment and Exploiting Spectrum Sensing Data Falsification Attack Using Stochastic Game Theory in Wireless Cognitive Radio Network. Wirel. Pers. Commun. 2017, 102, 799–816. [Google Scholar] [CrossRef]
- Subbulakshmi, P.; Prakash, M. Mitigating eavesdropping by using fuzzy based MDPOP-Q learning approach and multilevel Stackelberg game theoretic approach in wireless CRN. Cogn. Syst. Res. 2018, 52, 853–861. [Google Scholar] [CrossRef]
- Mohan, P.; Sundaram, M.; Satpathy, S.; Das, S. An efficient technique for cloud storage using secured de-duplication algorithm. J. Intell. Fuzzy Syst. 2021, 41, 2969–2980. [Google Scholar] [CrossRef]
- Jain, D.K.; Liu, X.; Neelakandan, S.; Prakash, M. Modeling of human action recognition using hyperparameter tuned deep learning model. J. Electron. Imaging 2022, 32, 011211. [Google Scholar] [CrossRef]
- Prakash, P.R.; Anuradha, D.; Iqbal, J.; Galety, M.G.; Singh, R.; Neelakandan, S. A novel convolutional neural network with gated recurrent unit for automated speech emotion recognition and classification. J. Control Decis. 2022, 1–10. [Google Scholar] [CrossRef]
- Neelakandan, S.; Prakash, M.; Bhargava, S.; Mohan, K.; Robert, N.R.; Upadhye, S. Optimal Stacked Sparse Autoencoder Based Traffic Flow Prediction in Intelligent Transportation Systems. In Virtual and Augmented Reality for Automobile Industry: Innovation Vision and Applications; Springer: Cham, Switzerland, 2022; pp. 111–127. [Google Scholar] [CrossRef]
- Banu, J.F.; Neelakandan, S.; Geetha, B.; Selvalakshmi, V.; Umadevi, A.; Martinson, E.O. Artificial Intelligence Based Customer Churn Prediction Model for Business Markets. Comput. Intell. Neurosci. 2022, 2022, 1703696. [Google Scholar] [CrossRef]
- Neelakandan, S.; Prakash, M.; Geetha, B.; Nanda, A.K.; Metwally, A.M.; Santhamoorthy, M.; Gupta, M.S. Metaheuristics with Deep Transfer Learning Enabled Detection and classification model for industrial waste management. Chemosphere 2022, 308, 136046. [Google Scholar] [CrossRef]
- Sreekala, K.; Cyril, C.P.D.; Neelakandan, S.; Chandrasekaran, S.; Walia, R.; Martinson, E.O. Capsule Network-Based Deep Transfer Learning Model for Face Recognition. Wirel. Commun. Mob. Comput. 2022, 2022, 2086613. [Google Scholar] [CrossRef]
- Jain, D.K.; Neelakandan, S.; Veeramani, T.; Bhatia, S.; Memon, F.H. Design of fuzzy logic based energy management and traffic predictive model for cyber physical systems. Comput. Electr. Eng. 2022, 102, 108135. [Google Scholar] [CrossRef]
- Raghavendra, S.; Harshavardhan, A.; Neelakandan, S.; Partheepan, R.; Walia, R.; Rao, V.C.S. Multilayer Stacked Probabilistic Belief Network-Based Brain Tumor Segmentation and Classification. Int. J. Found. Comput. Sci. 2022, 33, 559–582. [Google Scholar] [CrossRef]
- Parthiban, S.; Harshavardhan, A.; Neelakandan, S.; Prashanthi, V.; Alolo, A.-R.A.A.; Velmurugan, S. Chaotic Salp Swarm Optimization-Based Energy-Aware VMP Technique for Cloud Data Centers. Comput. Intell. Neurosci. 2022, 2022, 4343476. [Google Scholar] [CrossRef]
- Jain, D.K.; Tyagi, S.K.S.; Neelakandan, S.; Prakash, M.; Natrayan, L. Metaheuristic Optimization-Based Resource Allocation Technique for Cybertwin-Driven 6G on IoE Environment. IEEE Trans. Ind. Inform. 2021, 18, 4884–4892. [Google Scholar] [CrossRef]
- Venu, D.; Mayuri, A.; Neelakandan, S.; Murthy, G.; Arulkumar, N.; Shelke, N. An efficient low complexity compression based optimal homomorphic encryption for secure fiber optic communication. Optik 2021, 252, 168545. [Google Scholar] [CrossRef]
- Lakshmanna, K.; Subramani, N.; Alotaibi, Y.; Alghamdi, S.; Khalafand, O.I.; Nanda, A.K. Improved Metaheuristic-Driven Energy-Aware Cluster-Based Routing Scheme for IoT-Assisted Wireless Sensor Networks. Sustainability 2022, 14, 7712. [Google Scholar] [CrossRef]
- Harshavardhan, A.; Boyapati, P.; Neelakandan, S.; Akeji, A.A.A.-R.; Pundir, A.K.S.; Walia, R. LSGDM with Biogeography-Based Optimization (BBO) Model for Healthcare Applications. J. Healthc. Eng. 2022, 2022, 2170839. [Google Scholar] [CrossRef]
- Le, T.T.Q.; Tran, T.K.; Rege, M. Dynamic image for micro-expression recognition on region-based framework. In Proceedings of the 2020 IEEE 21st International Conference on Information Reuse and Integration for Data Science (IRI), Las Vegas, NV, USA, 11–13 August 2020; pp. 75–81. [Google Scholar]
- Saravanakumar, C.; Priscilla, R.; Prabha, B.; Kavitha, A.; Prakash, M.; Arun, C. An Efficient On-Demand Virtual Machine Migration in Cloud Using Common Deployment Model. Comput. Syst. Sci. Eng. 2022, 42, 245–256. [Google Scholar] [CrossRef]
- Geetha, B.T.; Mohan, P.; Mayuri, A.V.R.; Jackulin, T.; Stalin, J.L.A.; Anitha, V. Pigeon Inspired Optimization with Encryption Based Secure Medical Image Management System. Comput. Intell. Neurosci. 2022, 2022, 2243827. [Google Scholar] [CrossRef]
Reference & Year | Objectives | Classification | Significant Results | Accuracy Results |
---|---|---|---|---|
[20], 2021 | feature selection and classification methods for Facial Expression Recognition | Support Vector Machine, Random Forest and KNN algorithms | Based on minimum chi-square features, achieved a consistency performance of many controlled classifiers to determine face expression. | Achieved a 94.23% accuracy. |
[21], 2021 | To propose effective classification Sequence of face and Expression collection | Random forest, Decision Tree, SVM and KNN algorithms | Reliever-F technique for function by focusing on the utilization of a small number of attributes. | Achieved a 94.93% accuracy. |
[22], 2021 | To propose efficient modality fusion | Fuzzy Fusion based neural networks | Imbalanced emotion recognition is handled by TSFFCNN | Achieved eNTERFACE’ 05 90.82% |
[23], 2020 | To improve the spontaneous detection of facial micro-expressions by sophisticated hand extraction model. | Convolutional Neural Networks algorithms | Simple methods and effective classification for micro expression | Achieved 67.3% for SMIC dataset, Achieved 66.67% SAMM dataset |
Label | Description | No. of Images |
---|---|---|
An | Anger | 45 |
Co | Contempt | 18 |
Di | Disgust | 59 |
Fe | Fear | 25 |
Ha | Happy | 69 |
Nu | Neutral | 593 |
Sa | Sad | 28 |
Total Number of Images | 837 |
Labels | Accuracy | Sensitivity | Specificity | F-Score | MCC |
---|---|---|---|---|---|
Training Validation (70%) | |||||
An | 99.32 | 93.33 | 99.64 | 93.33 | 92.97 |
Co | 98.63 | 46.15 | 99.83 | 60.00 | 62.33 |
Di | 98.97 | 93.02 | 99.45 | 93.02 | 92.47 |
Fe | 99.15 | 78.57 | 99.65 | 81.48 | 81.10 |
Ha | 98.63 | 90.74 | 99.44 | 92.45 | 91.72 |
Nu | 97.95 | 99.76 | 93.64 | 98.56 | 95.07 |
Sa | 99.49 | 89.47 | 99.82 | 91.89 | 91.66 |
Average | 98.88 | 84.44 | 98.78 | 87.25 | 86.76 |
Testing (30%) | |||||
An | 100.00 | 100.00 | 100.00 | 100.00 | 100.00 |
Co | 99.60 | 80.00 | 100.00 | 88.89 | 89.26 |
Di | 99.21 | 93.75 | 99.58 | 93.75 | 93.33 |
Fe | 99.60 | 90.91 | 100.00 | 95.24 | 95.15 |
Ha | 99.60 | 93.33 | 100.00 | 96.55 | 96.41 |
Nu | 97.22 | 98.90 | 92.96 | 98.08 | 93.09 |
Sa | 99.21 | 88.89 | 99.59 | 88.89 | 88.48 |
Average | 99.21 | 92.25 | 98.87 | 94.49 | 93.67 |
Labels | Accuracy | Sensitivity | Specificity | F-Score | MCC |
---|---|---|---|---|---|
Training Phase (80%) | |||||
An | 99.10 | 89.74 | 99.68 | 92.11 | 91.67 |
Co | 99.25 | 66.67 | 100.00 | 80.00 | 81.34 |
Di | 98.95 | 91.11 | 99.52 | 92.13 | 91.58 |
Fe | 99.40 | 75.00 | 100.00 | 85.71 | 86.34 |
Ha | 98.21 | 89.66 | 99.02 | 89.66 | 88.67 |
Nu | 97.31 | 99.36 | 92.39 | 98.12 | 93.50 |
Sa | 99.10 | 87.50 | 99.53 | 87.50 | 87.03 |
Average | 98.76 | 85.58 | 98.59 | 89.32 | 88.59 |
Testing Phase (20%) | |||||
An | 100.00 | 100.00 | 100.00 | 100.00 | 100.00 |
Co | 99.40 | 100.00 | 99.39 | 85.71 | 86.34 |
Di | 98.81 | 100.00 | 98.70 | 93.33 | 92.93 |
Fe | 98.21 | 66.67 | 100.00 | 80.00 | 80.89 |
Ha | 98.81 | 90.91 | 99.36 | 90.91 | 90.27 |
Nu | 97.62 | 99.17 | 93.62 | 98.36 | 94.06 |
Sa | 98.81 | 50.00 | 100.00 | 66.67 | 70.28 |
Average | 98.81 | 86.68 | 98.73 | 87.85 | 87.82 |
Methods | Accuracy (%) |
---|---|
RFER-EADL | 99.21 |
LLDHF-FER | 88.49 |
DSA-FER | 89.64 |
FD-CNN | 94.35 |
LSTM | 93.12 |
Bi-LSTM | 93.87 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Arul Vinayakam Rajasimman, M.; Manoharan, R.K.; Subramani, N.; Aridoss, M.; Galety, M.G. Robust Facial Expression Recognition Using an Evolutionary Algorithm with a Deep Learning Model. Appl. Sci. 2023, 13, 468. https://doi.org/10.3390/app13010468
Arul Vinayakam Rajasimman M, Manoharan RK, Subramani N, Aridoss M, Galety MG. Robust Facial Expression Recognition Using an Evolutionary Algorithm with a Deep Learning Model. Applied Sciences. 2023; 13(1):468. https://doi.org/10.3390/app13010468
Chicago/Turabian StyleArul Vinayakam Rajasimman, Mayuri, Ranjith Kumar Manoharan, Neelakandan Subramani, Manimaran Aridoss, and Mohammad Gouse Galety. 2023. "Robust Facial Expression Recognition Using an Evolutionary Algorithm with a Deep Learning Model" Applied Sciences 13, no. 1: 468. https://doi.org/10.3390/app13010468