广西师范大学学报（自然科学版） ›› 2017, Vol. 35 ›› Issue (3): 45-52.doi: 10.16088/j.issn.1001-6600.2017.03.006

• • 上一篇    下一篇

基于对比度金字塔图像融合的自发笑脸识别

夏海英*, 喻潇琪   

  1. 广西师范大学电子工程学院,广西桂林541004
  • 出版日期:2017-07-25 发布日期:2018-07-25
  • 通讯作者: 夏海英(1983—),女,山东聊城人,广西师范大学副教授,博士。E-mail:xhyhust@gmail.com
  • 基金资助:
    广西自动检测技术及仪器重点实验室开发基金 (YQ1402); 广西多源信息挖掘与安全重点实验室开发基金 (MIMS15-05);广西药用资源化学与药物分子工程重点实验室开发基金(CMEMR2014-B15);国家星火计划重点项目(2015GA790002)

Spontaneous Smile Recognition Based on CPD Image Fusion

XIA Haiying*, YU Xiaoqi   

  1. College of Electronic Engineering, Guangxi Normal University, Guilin Guangxi 541004,China
  • Online:2017-07-25 Published:2018-07-25

PDF (PC)

251

摘要： 自发笑脸识别是表情识别研究领域的重要部分,在诸多领域均有着广泛应用。本文提出一种基于对比度金字塔(contrast pyramid decomposition,CPD)图像融合的自发笑脸识别方法。首先将2种光源的图像分别进行4层Gauss塔式分解,建立对应的对比度金字塔,并对每层对应的对比度金字塔采用像素灰度平均法融合规则进行融合,得到融合图像,最后分别提取融合图像的LBP(local binary pattern)特征和LDP(local directional pattern)特征,使用SVM分类器进行分类识别。实验结果表明:使用本文方法融合图像作为识别对象,提取LBP特征和LDP特征的笑脸识别率分别达到96.69%和97.19%,总识别率分别达到96.51%和96.78%,明显优于其他算法,表明本文融合算法提升了自发笑脸识别的识别性能。

关键词: 图像融合, 对比度金字塔分解, 笑脸识别, LBP特征, LDP特征

Abstract: Spontaneous smile recognition is an important part in the field of facial expression recognition, which is widely used in many fields. This paper presents a spontaneous smile recognition method by fusing infrared and visible images based on contrast pyramid decomposition(CPD). Infrared and visible images are decomposed with 4 layers Gauss Pyramids respectively, and the corresponding contrast pyramids are built. Fusion image which is obtained by combining each layer of infrared image and visible image based on the fusion rule—pixel average. Then LBP features and LDP features are extracted from images, and smile and not smile expressions are classified by SVM classifier. The experimental results show that the smile recognition rates with the fusion images based on LBP features and LDP features can reach 96.69% and 97.19% respectively,and the overall recognition rates with the fusion images based on LBP features and LDP features can reach 96.51% and 96.78% respectively. These results are significantly better than the results with other methods. The results show that the proposed algorithm can improve the recognition performance of the spontaneous smile recognition.

Key words: image fusion, contrast pyramid decomposition, smile recognition, LBP feature, LDP feature

中图分类号: 

  • TP391.4
[1] BARON-COHEN S, RING H A, BULLMORE E T, et al. The amygdala theory of autism[J].Neuroscience and Biobehavioral Reviews, 2000, 24(3): 355-364.
[2] CUI Dongshun, HUANG Guangbin, LIU Tianchi. Smile detection using pair-wise distance vector and extreme learning machine[C]//Proceeding of the 2016 International Joint Conference on Neural Networks. Piscataway, NJ: IEEE Press, 2016: 2298-2305. DOI: 10.1109/IJCNN.2016.7727484.
[3] ZENG Zhihong, PANTIC M, ROISMAN G I, et al. A survey of affect recognition methods: audio, visual, and spontaneous expressions[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(1): 39-58. DOI: 10.1109/TPAMI.2008.52.
[4] SUN Yanjia, AKANSU A N. Automatic inference of mental states from spontaneous facial expressions[C]//Proceeding of the 2014 IEEE International Conference on Acoustic, Speech and Signal Processing. Piscataway, NJ: IEEE Press,2014:719-723.DOI:10.1109/ICASSP.2014.6853690.
[5] ASHRAF A B, LUCEY S, COHN J F, et al. The painful face-pain expression recognition using active appearance models[J].Image and Vision Computing,2009, 27(1): 1788-1796. DOI: 10.1016/j.imavis.2009.05.007.
[6] UGLI I S Z, UGLI B M M. Optimization detection of smiling and opening eyes in faces with algorithm LBP[C]// Proceeding of the 2016 IEEE International Conference on Information Science and Communications Technologies. Piscataway, NJ: IEEE Press, 2016: 1-4. DOI: 10.1109/ICISCT.2016.7777380.
[7] 何俊, 何忠文, 蔡建峰, 等.自发表情识别方法综述[J].计算机应用研究, 2016, 33(1): 12-16. DOI: 10.3969/j.issn.1001-3695.2016.01.003.
[8] FASEL B, LUETTIN J. Automatic facial expression analysis: a survey[J]. Pattern Recognition, 2003, 36(1): 259-275.
[9] ADINI Y, MOSES Y, ULLMAN S. Face recognition: the problem of compensating for changes in illumination direction[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19(7): 721-732. DOI: 10.1109/34.598229.
[10] KHAM M M, WARD R D, INGLEBY M. Automated classification and recognition of facial expressions using infrared thermal imaging[C]//Proceeding of the 2004 IEEE Conference on Cybernetics and Intelligent Systems. Piscataway, NJ: IEEE Press, 2004: 202-206. DOI: 10.1109/ICCIS.2004.1460412.
[11] YOSHITOMI Y, KIM S I, KAWANO T, et al. Effects of sensor fusion for recognition of emotional states using voice, face image and thermal image of face[C]//Proceedings of the 9th IEEE International Workshop on Robot and Human Interactive Communication. Piscataway, NJ: IEEE Press, 2000:178-183. DOI: 10.1109/ROMAN.2000.892491.
[12] WANG Zhaoyu, WANG Shangfei. Spontaneous facial expression recognition by using feature-level fusion of visible and thermal infrared images[C]//Proceeding of the 2011 IEEE International Workshop on Machine Learning for Signal Processing. Piscataway, NJ: IEEE Press, 2011: 1-6. DOI: 10.1109/MLSP.2011.6064564.
[13] 夏海英, 徐鲁辉.基于差分纹理的人脸表情识别[J].计算机应用研究, 2015,32(11): 3504-3507. DOI: 10.3969/J.ISSN.1001-3695.2015.11.072.
[14] BURT P, ADELSON E. The laplacian pyramid as a compact image code[J]. IEEE Transactions on Communications, 1983, 31(4): 532-540. DOI: 10.1109/TCOM.1983.1095851.
[15] PRASAD L, IYENGAR S S. Wavelet analysis with applications to image proceeding[M]. Boca Raton, Florida: CRC Press, 1997:217-222.
[16] OJALA T, PIETIKINEN M, HARWOOD D. A comparative study of texture measures with classification based on featured distributions[J]. Pattern Recognition, 1996, 29(1):51-59.
[17] JABID T, KABIR M H, CHAE O. Local directional pattern (LDP)- a robust image descriptor for object recognition[C]//Proceeding of the 7th Seventh IEEE International Conference on Advanced Video and Signal Based Surveillance. Piscataway, NJ: IEEE Press, 2010: 482-487. DOI:10.1109/AVSS.2010.17.
[18] 夏海英, 徐鲁辉.基于主动形状模型差分纹理和局部方向模式特征融合的人脸表情识别[J].计算机应用, 2015, 35(3): 783-786, 796. DOI: 10.11772/j.issn.1001-9081.2015.03.783.
[19] WANG Shangfei, LIU Zhilei, LV Siliang, et al. A natural visible and infrared facial expression database for expression recognition and emotion inference[J].IEEE Transactions on Multimedia, 2010, 12(7): 682-691. DOI: 10.1109/TMM.2010.2060716.
[20] CHEN Jinhui, TAKIGUCHI T, ARIKI Y. Facial expression recognition with multithreaded cascade of rotation-invariant HOG[C]//Proceeding of the 2015 International Conference on Affective Computing and Intelligent Interaction. Piscataway, NJ: IEEE Press, 2015: 636-642. DOI: 10.1109/ACII.2015.7344636.
[21] ABDULRAHMAN M, GWADABE T R, ABDU F J, et al. Gabor wavelet transform based facial expression recognition using PCA and LBP[C]//Proceeding of the 22nd Signal Processing and Communications Applications Conference. Piscataway, NJ: IEEE Press, 2014: 2265-2268. DOI: 10.1109/SUI.2014.6830717.
[22] AN Shu, RUAN Qiuqi. 3D facial expression recognition algorithm using local threshold binary pattern and histogram of oriented gradient[C]//Proceeding of the 2016 IEEE 13th International Conference on Signal Processing. Piscataway, NJ: IEEE Press, 2016: 265-270. DOI: 10.1109/ICSP.2016.7877838.
[1] 薛洋,曾庆科,夏海英,王文涛. 基于卷积神经网络超分辨率重建的遥感图像融合[J]. 广西师范大学学报（自然科学版）, 2018, 36(2): 33-41.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 广西师范大学学报（自然科学版）编辑部
地址：广西桂林市三里店育才路15号　邮编：541004
电话：0773-5857325　E-mail: gxsdzkb@mailbox.gxnu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发