Next Article in Journal
ETFT: Equiangular Tight Frame Transformer for Imbalanced Semantic Segmentation
Previous Article in Journal
BGF-YOLOv10: Small Object Detection Algorithm from Unmanned Aerial Vehicle Perspective Based on Improved YOLOv10
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 24
Issue 21
10.3390/s24216912
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Facial Expression Recognition-You Only Look Once-Neighborhood Coordinate Attention Mamba: Facial Expression Detection and Classification Based on Neighbor and Coordinates Attention Mechanism

by
Cheng Peng
1,†,
Mingqi Sun
2,†,
Kun Zou
1,*,
Bowen Zhang
3,
Genan Dai
3 and
Ah Chung Tsoi
4
1
School of Computing, Zhongshan Institute, University of Electronic Science and Technology of China, Zhongshan 528402, China
2
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
3
College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China
4
School of Computing and Information Technology, University of Wollongong, Wollongong, NSW 2522, Australia
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Sensors 2024, 24(21), 6912; https://doi.org/10.3390/s24216912
Submission received: 10 August 2024 / Revised: 12 September 2024 / Accepted: 22 October 2024 / Published: 28 October 2024
(This article belongs to the Section Sensing and Imaging)
Browse Figures
Versions Notes

Abstract

In studying the joint object detection and classification problem for facial expression recognition (FER) deploying the YOLOX framework, we introduce a novel feature extractor, called neighborhood coordinate attention Mamba (NCAMamba) to substitute for the original feature extractor in the Feature Pyramid Network (FPN). NCAMamba combines the background information reduction capabilities of Mamba, the local neighborhood relationship understanding of neighborhood attention, and the directional relationship understanding of coordinate attention. The resulting FER-YOLO-NCAMamba model, when applied to two unaligned FER benchmark datasets, RAF-DB and SFEW, obtains significantly improved mean average precision (mAP) scores when compared with those obtained by other state-of-the-art methods. Moreover, in ablation studies, it is found that the NCA module is relatively more important than the Visual State Space (VSS), a version of using Mamba for image processing, and in visualization studies using the grad-CAM method, it reveals that regions around the nose tip are critical to recognizing the expression; if it is too large, it may lead to erroneous prediction, while a small focused region would lead to correct recognition; this may explain why FER of unaligned faces is such a challenging problem.
Keywords: facial expression recognition; visual state space model; attention; object detection facial expression recognition; visual state space model; attention; object detection

Share and Cite

MDPI and ACS Style

Peng, C.; Sun, M.; Zou, K.; Zhang, B.; Dai, G.; Tsoi, A.C. Facial Expression Recognition-You Only Look Once-Neighborhood Coordinate Attention Mamba: Facial Expression Detection and Classification Based on Neighbor and Coordinates Attention Mechanism. Sensors 2024, 24, 6912. https://doi.org/10.3390/s24216912

AMA Style

Peng C, Sun M, Zou K, Zhang B, Dai G, Tsoi AC. Facial Expression Recognition-You Only Look Once-Neighborhood Coordinate Attention Mamba: Facial Expression Detection and Classification Based on Neighbor and Coordinates Attention Mechanism. Sensors. 2024; 24(21):6912. https://doi.org/10.3390/s24216912

Chicago/Turabian Style

Peng, Cheng, Mingqi Sun, Kun Zou, Bowen Zhang, Genan Dai, and Ah Chung Tsoi. 2024. "Facial Expression Recognition-You Only Look Once-Neighborhood Coordinate Attention Mamba: Facial Expression Detection and Classification Based on Neighbor and Coordinates Attention Mechanism" Sensors 24, no. 21: 6912. https://doi.org/10.3390/s24216912

APA Style

Peng, C., Sun, M., Zou, K., Zhang, B., Dai, G., & Tsoi, A. C. (2024). Facial Expression Recognition-You Only Look Once-Neighborhood Coordinate Attention Mamba: Facial Expression Detection and Classification Based on Neighbor and Coordinates Attention Mechanism. Sensors, 24(21), 6912. https://doi.org/10.3390/s24216912

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop