Journal of Guangxi Normal University(Natural Science Edition) ›› 2025, Vol. 43 ›› Issue (2): 70-82.doi: 10.16088/j.issn.1001-6600.2024021601

• Intelligence Information Processing • Previous Articles     Next Articles

Facial Expression Recognition Based on Noise-Resistant Dual Constraint Network

SU Chunhai1,2, XIA Haiying1,2*   

  1. 1. Guangxi Key Laboratory of Brain-inspired Computing and Intelligent Chips (Guangxi Normal University), Guilin Guangxi 541004, China;
    2. School of Electronic and Information Engineering/School of Integrated Circuits, Guangxi Normal University, Guilin Guangxi 541004, China
  • Received:2024-02-16 Revised:2024-04-15 Online:2025-03-05 Published:2025-04-02

Abstract: Noise is inevitably present in datasets due to labeling subjectivity, image blurring, and other factors, making expression recognition more challenging. Existing facial expression recognition methods typically address noisy labels by partially overfitting to them. In this paper, a novel Noise-Resistant Dual Constraint Network (NDC-Net) is proposed to automatically suppress noisy samples. NDC-Net primarily consists of two constraint mechanisms: Class Activation mapping attention Consistency (CAC) and Channel and Spatial feature Consistency (CSC). CAC is used to make the model focus on locally important feature information and reduces the overfitting to noisy labels, while CSC is used to ensure that the model emphasizes task-relevant information from both channels and spatial dimensions during feature extraction, ignoring irrelevant information, and reducing reliance on noisy labels. Additionally, to enhance the performance of NDC-Net, input samples are augmented with strategies such as rotation and scaling. NDC-Net achieves recognition performances of 86.57%, 88.22%, and 59.78% under 30% label noise for RAF-DB, FERPlus, and AffectNet datasets, respectively. These results significantly outperform the state-of-the-art noisy labeling methods, such as EAC, NCCTFER. Moreover, NDC-Net also shows strong generalisation capability on general classification datasets such as CIFAR100 and Tiny-ImageNet.

Key words: noisy label learning, facial expression recognition, deep learning, supervised learning, attention mechanisms

CLC Number:  TP391.41
[1] 李俊侠,张秦,郑桂妹.超宽带雷达人体姿态识别综述[J].计算机工程与应用,2021,57(3):14-23.DOI: 10.3778/j.issn.1002-8331.2009-0444.
[2] CHEN S K, WANG J F, CHEN Y D, et al. Label distribution learning on auxiliary label space graphs for facial expression recognition[C] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2020: 13981-13990. DOI: 10.1109/CVPR42600.2020.01400.
[3] FAN X Y, DENG Z Y, WANG K, et al. Learning discriminative representation for facial expression recognition from uncertainties[C] // 2020 IEEE International Conference on Image Processing (ICIP). Los Alamitos, CA: IEEE Computer Society, 2020: 903-907. DOI: 10.1109/ICIP40778.2020.9190643.
[4] 王晓峰,王昆,刘轩,等.自适应重加权池化深度多任务学习的表情识别[J].计算机工程与设计,2022,43(4): 1111-1120.DOI: 10.16208/j.issn1000-7024.2022.04.029.
[5] SHE J H, HU Y B, SHI H L, et al. Dive into ambiguity: latent distribution mining and pairwise uncertainty estimation for facial expression recognition[C] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2021: 6244-6253. DOI: 10.1109/CVPR46437.2021.00618.
[6 ] WANG K, PENG X J, YANG J F, et al. Suppressing uncertainties for large-scale facial expression recognition[C] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2020: 6896-6905. DOI: 10.1109/CVPR42600.2020.00693.
[7] ZHANG Y H, WANG C R, DENG W H. Relative uncertainty learning for facial expression recognition[C] // Advances in Neural Information Processing Systems 34 (NeurIPS 2021). Red Hook, NY: Curran Associates Inc., 2021: 17616-17627.
[8] MA F Y, SUN B, LI S T. Transformer-augmented network with online label correction for facial expression recognition[J]. IEEE Transactions on Affective Computing, 2024, 15(2): 593-605. DOI: 10.1109/TAFFC.2023.3285231.
[9] GERA D, KUMAR B N S, KUMAR B V R, et al. Class adaptive threshold and negative class guided noisy annotation robust Facial Expression Recognition[EB/OL]. (2023-05-03)[2024-02-16]. https://arxiv.org/abs/2305.01884. DOI: 10.48550/arXiv.2305.01884.
[10] ZHANG Y H, WANG C R, LING X, et al. Learn from all: erasing attention consistency for noisy label facial expression recognition[C] // Computer Vision-ECCV 2022. Cham: Springer, 2022: 418-434. DOI: 10.1007/978-3-031-19809-0_24.
[11] ZHOU B L, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2016: 2921-2929. DOI: 10.1109/CVPR.2016.319.
[12] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C] // Computer Vision-ECCV 2018. Cham: Springer, 2018: 3-19. DOI: 10.1007/978-3-030-01234-2_1.
[13] 高丹,陈建英,谢盈.A-PSPNet:一种融合注意力机制的PSPNet图像语义分割模型[J].中国电子科学研究院学报,2020,15(6):518-523.DOI: 10.3969/j.issn.1673-5692.2020.06.005.
[14] 杨军奇,冯全,王书志,等.基于改进YOLOv4的田间密集小目标检测方法[J].东北农业大学学报,2022,53(5):69-79.DOI: 10.3969/j.issn.1005-9369.2022.05.008.
[15] 张珂,冯晓晗,郭玉荣,等.图像分类的深度卷积神经网络模型综述[J].中国图象图形学报,2021,26(10):2305-2325.DOI: 10.11834/jig.200302.
[16] 马龙祥,杨浩,宋婷婷,等.基于高分辨率特征的舌象分割算法研究[J].计算机工程,2020,46(10):248-252.DOI: 10.19678/j.issn.1000-3428.0056685.
[17] LI S, DENG W H, DU J P. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild[C] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2017: 2584-2593. DOI: 10.1109/CVPR.2017.277.
[18] BARSOUM E, ZHANG C, FERRER C C, et al. Training deep networks for facial expression recognition with crowd-sourced label distribution[C] // Proceedings of the 18th ACM International Conference on Multimodal Interaction. New York, NY: Association for Computing Machinery, 2016: 279-283. DOI: 10.1145/2993148.2993165.
[19] MOLLAHOSSEINI A, HASANI B, MAHOOR M H. AffectNet: a database for facial expression, valence, and arousal computing in the wild[J]. IEEE Transactions on Affective Computing, 2019, 10(1): 18-31. DOI: 10.1109/TAFFC.2017.2740923.
[20] KRIZHEVSKY A. Learning multiple layers of features from tiny images[D]. Toronto: University of Toronto, 2009.
[21] RUSSAKOVSKY O, DENG J, SU H, et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252. DOI: 10.1007/s11263-015-0816-y.
[22] 于婉莹,梁美玉,王笑笑,等.基于深度注意力网络的课堂教学视频中学生表情识别与智能教学评估[J].计算机应用,2022,42(3):743-749.DOI: 10.11772/j.issn.1001-9081.2021040846.
[23] 麻永田,齐晶,张秋实,等.基于二阶统计量的小样本学习算法研究[J].北京联合大学学报(自然科学版),2021,35(4):73-78.DOI: 10.16255/j.cnki.ldxbz.2021.04.012.
[24] 李成范,胡子荣,刘岚,等.一种基于“中国视云”平台的CNN核函数可视化方法[J].实验室研究与探索,2021,40(5):57-61.DOI: 10.19927/j.cnki.syyt.2021.05.014.
[25] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2016: 770-778. DOI: 10.1109/CVPR.2016.90.
[26] GUO Y D, ZHANG L, HU Y X, et al. MS-Celeb-1M: a dataset and benchmark for large-scale face recognition[C] // Computer Vision-ECCV 2016. Cham: Springer, 2016: 87-102. DOI: 10.1007/978-3-319-46487-9_6.
[27] KINGMA D, BA J. Adam: a method for stochastic optimization[EB/OL]. (2014-12-22)[2024-02-16]. https://arxiv.org/abs/1412.6980v1. DOI: 10.48550/arXiv.1412.6980.
[28] LI Z Y, ARORA S. An exponential learning rate schedule for deep learning[EB/OL]. (2019-11-21)[2024-02-16]. https://arxiv.org/abs/1910.07454. DOI: 10.48550/arXiv.1910.07454.
[29] LE N, NGUYEN K, TRAN Q, et al. Uncertainty-aware label distribution learning for facial expression recognition[C] // 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Los Alamitos, CA: IEEE Computer Society, 2023: 6077-6086. DOI: 10.1109/WACV56688.2023.00603.
[30] ZENG J B, SHAN S G, CHEN X L. Facial expression recognition with inconsistently annotated datasets[C] // Computer Vision-ECCV 2018. Cham: Springer, 2018: 227-243. DOI: 10.1007/978-3-030-01261-8_14.
[31] WANG K, PENG X J, YANG J F, et al. Region attention networks for pose and occlusion robust facial expression recognition[J]. IEEE Transactions on Image Processing, 2020, 29: 4057-4069. DOI: 10.1109/TIP.2019.2956143.
[32] FARZANEH A H, QI X J. Facial expression recognition in the wild via deep attentive center loss [C] // 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). Los Alamitos, CA: IEEE Computer Society, 2021: 2401-2410. DOI: 10.1109/WACV48630.2021.00245.
[33] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2018: 7132-7141. DOI: 10.1109/CVPR.2018.00745.
[34] WANG Q L, WU B G, ZHU P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2020: 11531-11539. DOI: 10.1109/ CVPR42600.2020.01155.
[35] GU Y, YAN H, ZHANG X, et al. Toward facial expression recognition in the wild via noise-tolerant network[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(5): 2033-2047. DOI: 10.1109/TCSVT.2022.3220669.
[1] GUO Xiangyu, SHI Tianyi, CHEN Yannan, NAN Xinyuan, CAI Xin. Research on Foreign Object Detection in Railway Overhead Contact System Based on YOLO-CDBW Model [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(2): 56-69.
[2] LI Xin, NING Jing. Online Assessment of Transient Stability in Power Systems Based on Spatiotemporal Feature Fusion [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 89-100.
[3] HOU Haiyan, TAN Yumei, SONG Shuxiang, XIA Haiying. Head Pose-Robust Facial Expression Recognition [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 126-137.
[4] LU Jiahui, CHEN Qingfeng, WANG Wenguang, YU Qian, HE Naixu, HAN Zongzhao. Multi-scale Attention Learning for Abdomen Multi-organ Image Segmentation [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 138-148.
[5] DU Shuaiwen, JIN Ting. A Deep Hybrid Recommendation Algorithm Based on User Behavior Characteristics [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(5): 91-100.
[6] LI Xiangli, MEI Jianping, MO Yuanjian. Adaptive Semi-supervised Multi-view Clustering Based on Hypergraph Regular NMF [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(4): 137-152.
[7] YI Jianbing, PENG Xin, CAO Feng, LI Jun, XIE Weijia. Research on Point Cloud Registration Algorithm Based on Multi-scale Feature Fusion [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(3): 108-120.
[8] XIAO Yuting, LÜ Xiaoqi, GU Yu, LIU Chuanqiang. Classification of Diabetic Retinopathy Based on Split Residual Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(1): 91-101.
[9] GAO Fei, GUO Xiaobin, YUAN Dongfang, CAO Fujun. Improved PINNs Method for Solving the Convective Dominant Diffusion Equation with Boundary Layer [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 33-50.
[10] SONG Guanwu, CHEN Zhiming, LI Jianjun. Remote Sensing Image Classification with Cascade Attention Based on ResNet-50 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 80-91.
[11] JIANG Yibo, LIU Huijia, WU Tian. Research on Identification of Lightning Overvoltage in Transmission Line by Improved Residual Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(4): 74-83.
[12] YANG Shuozhen, ZHANG Long, WANG Jianhua, ZHANG Hengyuan. Review of Sound Event Detection [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 1-18.
[13] WANG Luna, DU Hongbo, ZHU Lijun. Stacked Capsule Autoencoders Optimization Algorithm Based on Manifold Regularization [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 76-85.
[14] ZHANG Ping, XU Qiaozhi. Segmentation of Lung Nodules Based on Multi-receptive Field and Grouping Attention Mechanism [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 76-87.
[15] LI Yongjie, ZHOU Guihong, LIU Bo. Fusion Algorithm of Face Detection and Head Pose Estimation Based on YOLOv3 Model [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 95-103.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!