Journal of Guangxi Normal University(Natural Science Edition) ›› 2017, Vol. 35 ›› Issue (3): 45-52.doi: 10.16088/j.issn.1001-6600.2017.03.006

Previous Articles     Next Articles

Spontaneous Smile Recognition Based on CPD Image Fusion

XIA Haiying*, YU Xiaoqi   

  1. College of Electronic Engineering, Guangxi Normal University, Guilin Guangxi 541004,China
  • Online:2017-07-25 Published:2018-07-25

Abstract: Spontaneous smile recognition is an important part in the field of facial expression recognition, which is widely used in many fields. This paper presents a spontaneous smile recognition method by fusing infrared and visible images based on contrast pyramid decomposition(CPD). Infrared and visible images are decomposed with 4 layers Gauss Pyramids respectively, and the corresponding contrast pyramids are built. Fusion image which is obtained by combining each layer of infrared image and visible image based on the fusion rule—pixel average. Then LBP features and LDP features are extracted from images, and smile and not smile expressions are classified by SVM classifier. The experimental results show that the smile recognition rates with the fusion images based on LBP features and LDP features can reach 96.69% and 97.19% respectively,and the overall recognition rates with the fusion images based on LBP features and LDP features can reach 96.51% and 96.78% respectively. These results are significantly better than the results with other methods. The results show that the proposed algorithm can improve the recognition performance of the spontaneous smile recognition.

Key words: image fusion, contrast pyramid decomposition, smile recognition, LBP feature, LDP feature

CLC Number: 

  • TP391.4
[1] BARON-COHEN S, RING H A, BULLMORE E T, et al. The amygdala theory of autism[J].Neuroscience and Biobehavioral Reviews, 2000, 24(3): 355-364.
[2] CUI Dongshun, HUANG Guangbin, LIU Tianchi. Smile detection using pair-wise distance vector and extreme learning machine[C]//Proceeding of the 2016 International Joint Conference on Neural Networks. Piscataway, NJ: IEEE Press, 2016: 2298-2305. DOI: 10.1109/IJCNN.2016.7727484.
[3] ZENG Zhihong, PANTIC M, ROISMAN G I, et al. A survey of affect recognition methods: audio, visual, and spontaneous expressions[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(1): 39-58. DOI: 10.1109/TPAMI.2008.52.
[4] SUN Yanjia, AKANSU A N. Automatic inference of mental states from spontaneous facial expressions[C]//Proceeding of the 2014 IEEE International Conference on Acoustic, Speech and Signal Processing. Piscataway, NJ: IEEE Press,2014:719-723.DOI:10.1109/ICASSP.2014.6853690.
[5] ASHRAF A B, LUCEY S, COHN J F, et al. The painful face-pain expression recognition using active appearance models[J].Image and Vision Computing,2009, 27(1): 1788-1796. DOI: 10.1016/j.imavis.2009.05.007.
[6] UGLI I S Z, UGLI B M M. Optimization detection of smiling and opening eyes in faces with algorithm LBP[C]// Proceeding of the 2016 IEEE International Conference on Information Science and Communications Technologies. Piscataway, NJ: IEEE Press, 2016: 1-4. DOI: 10.1109/ICISCT.2016.7777380.
[7] 何俊, 何忠文, 蔡建峰, 等.自发表情识别方法综述[J].计算机应用研究, 2016, 33(1): 12-16. DOI: 10.3969/j.issn.1001-3695.2016.01.003.
[8] FASEL B, LUETTIN J. Automatic facial expression analysis: a survey[J]. Pattern Recognition, 2003, 36(1): 259-275.
[9] ADINI Y, MOSES Y, ULLMAN S. Face recognition: the problem of compensating for changes in illumination direction[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19(7): 721-732. DOI: 10.1109/34.598229.
[10] KHAM M M, WARD R D, INGLEBY M. Automated classification and recognition of facial expressions using infrared thermal imaging[C]//Proceeding of the 2004 IEEE Conference on Cybernetics and Intelligent Systems. Piscataway, NJ: IEEE Press, 2004: 202-206. DOI: 10.1109/ICCIS.2004.1460412.
[11] YOSHITOMI Y, KIM S I, KAWANO T, et al. Effects of sensor fusion for recognition of emotional states using voice, face image and thermal image of face[C]//Proceedings of the 9th IEEE International Workshop on Robot and Human Interactive Communication. Piscataway, NJ: IEEE Press, 2000:178-183. DOI: 10.1109/ROMAN.2000.892491.
[12] WANG Zhaoyu, WANG Shangfei. Spontaneous facial expression recognition by using feature-level fusion of visible and thermal infrared images[C]//Proceeding of the 2011 IEEE International Workshop on Machine Learning for Signal Processing. Piscataway, NJ: IEEE Press, 2011: 1-6. DOI: 10.1109/MLSP.2011.6064564.
[13] 夏海英, 徐鲁辉.基于差分纹理的人脸表情识别[J].计算机应用研究, 2015,32(11): 3504-3507. DOI: 10.3969/J.ISSN.1001-3695.2015.11.072.
[14] BURT P, ADELSON E. The laplacian pyramid as a compact image code[J]. IEEE Transactions on Communications, 1983, 31(4): 532-540. DOI: 10.1109/TCOM.1983.1095851.
[15] PRASAD L, IYENGAR S S. Wavelet analysis with applications to image proceeding[M]. Boca Raton, Florida: CRC Press, 1997:217-222.
[16] OJALA T, PIETIKINEN M, HARWOOD D. A comparative study of texture measures with classification based on featured distributions[J]. Pattern Recognition, 1996, 29(1):51-59.
[17] JABID T, KABIR M H, CHAE O. Local directional pattern (LDP)- a robust image descriptor for object recognition[C]//Proceeding of the 7th Seventh IEEE International Conference on Advanced Video and Signal Based Surveillance. Piscataway, NJ: IEEE Press, 2010: 482-487. DOI:10.1109/AVSS.2010.17.
[18] 夏海英, 徐鲁辉.基于主动形状模型差分纹理和局部方向模式特征融合的人脸表情识别[J].计算机应用, 2015, 35(3): 783-786, 796. DOI: 10.11772/j.issn.1001-9081.2015.03.783.
[19] WANG Shangfei, LIU Zhilei, LV Siliang, et al. A natural visible and infrared facial expression database for expression recognition and emotion inference[J].IEEE Transactions on Multimedia, 2010, 12(7): 682-691. DOI: 10.1109/TMM.2010.2060716.
[20] CHEN Jinhui, TAKIGUCHI T, ARIKI Y. Facial expression recognition with multithreaded cascade of rotation-invariant HOG[C]//Proceeding of the 2015 International Conference on Affective Computing and Intelligent Interaction. Piscataway, NJ: IEEE Press, 2015: 636-642. DOI: 10.1109/ACII.2015.7344636.
[21] ABDULRAHMAN M, GWADABE T R, ABDU F J, et al. Gabor wavelet transform based facial expression recognition using PCA and LBP[C]//Proceeding of the 22nd Signal Processing and Communications Applications Conference. Piscataway, NJ: IEEE Press, 2014: 2265-2268. DOI: 10.1109/SUI.2014.6830717.
[22] AN Shu, RUAN Qiuqi. 3D facial expression recognition algorithm using local threshold binary pattern and histogram of oriented gradient[C]//Proceeding of the 2016 IEEE 13th International Conference on Signal Processing. Piscataway, NJ: IEEE Press, 2016: 265-270. DOI: 10.1109/ICSP.2016.7877838.
[1] XUE Yang,ZENG Qingke,XIA Haiying,WANG Wentao. Remote Sensing Image Fusion Based on Convolutional Neural Network Super-resolution Reconstruction [J]. Journal of Guangxi Normal University(Natural Science Edition), 2018, 36(2): 33-41.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!