结合运动时序性的人脸表情识别方法

doi:10.3969/j.issn.0372-2112.2016.06.007

电子学报 ›› 2016, Vol. 44 ›› Issue (6): 1307-1313.DOI: 10.3969/j.issn.0372-2112.2016.06.007

结合运动时序性的人脸表情识别方法

邱玉, 赵杰煜, 汪燕芳

宁波大学信息科学与工程学院, 浙江宁波 315211

收稿日期:2014-08-08 修回日期:2015-05-17 出版日期:2016-06-25
- 作者简介:
- 邱玉女,1990年12月生于山东济宁,硕士研究生.研究方向:图像处理、模式识别.E-mail:qiuyu1204@126.com;赵杰煜 男,1965年11月生于浙江宁波,教授,博士生导师.研究方向:计算智能、模式识别、人机自然交互.E-mail:zhao-jieyu@nbu.edu.cn;汪燕芳 女,1989年1月生于安徽安庆,硕士研究生.研究方向:软件开发.
- 基金资助:
- 国家自然科学基金 (No.61571247); 科技部国际科技合作专项 (No.2013DFG12810,No.2012BAF12B11); 浙江省国际科技合作专项 (No.2013C24027); 浙江省自然科学基金 (No.LZ16F030001)

Facial Expression Recognition Using Temporal Relations Among Facial Movements

QIU Yu, ZHAO Jie-yu, WANG Yan-fang

Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo, Zhejiang 315211, China

Received:2014-08-08 Revised:2015-05-17 Online:2016-06-25 Published:2016-06-25
- Supported by:
- National Natural Science Foundation of China (No.61571247); International Science and Technology Cooperation Project of Ministry of Science and Technology (No.2013DFG12810, No.2012BAF12B11); International Science and Technology Cooperation Project of Zhejiang Province (No.2013C24027); National Natural Science Foundation of Zhejiang Province, China (No.LZ16F030001)

摘要/Abstract

摘要：

脸部肌肉之间的时空关系在人脸表情识别中起着重要作用,而当前的模型无法高效地捕获人脸的复杂全局时空关系使其未被广泛应用.为了解决上述问题,本文提出一种基于区间代数贝叶斯网络的人脸表情建模方法,该方法不仅能够捕获脸部的空间关系,也能捕获脸部的复杂时序关系,从而能够更加有效地对人脸表情进行识别.且该方法仅利用基于跟踪的特征且不需要手动标记峰值帧,可提高训练与识别的速度.在标准数据库CK+和MMI上进行实验发现本文方法在识别人脸表情过程中有效提高了准确率.

关键词: 表情识别, 脸部肌肉运动的时序性, 贝叶斯网络, 区间代数

Abstract:

Spatial and temporal relations between different facial muscles are very important in the facial expression recognition process.However, these implicit relations have not been widely used due to the limitation of the current models.In order to make full use of spatial and temporal information, we model the facial expression as a complex activity consisting of different facial events.Furthermore, we introduce a special Bayesian network to capture the temporal relations among facial events and develop the corresponding algorithm for facial expression modeling and recognition.We only use the features based on tracking results and this method does not require the peak frames, which can improve the speed of training and recognition.Experimental results on the benchmark databases CK+ and MMI show that the proposed method is feasible in facial expression recognition and considerably improves the recognition accuracy.

Key words: facial expression recognition, sequential facial events, Bayesian network, interval algebra

中图分类号:

TN391.4

邱玉, 赵杰煜, 汪燕芳. 结合运动时序性的人脸表情识别方法[J]. 电子学报, 2016, 44(6): 1307-1313.

QIU Yu, ZHAO Jie-yu, WANG Yan-fang . Facial Expression Recognition Using Temporal Relations Among Facial Movements[J]. Acta Electronica Sinica, 2016, 44(6): 1307-1313.

参考文献

[1] Pantic M,Rothkrantz L J.Expert system for automatic analysis of facial expressions[J].Image and Vision Computing,2000,18(11):881-905.
[2] Pantic M,Rothkrantz L J.Facial action recognition for facial expression analysis from static face images[J].Systems,Man,and Cybernetics,Part B:Cybernetics,IEEE Transactions on,2004,34(3):1449-1461.
[3] Buciu I,Kotropoulos C,Pitas I.ICA and Gabor representation for facial expression recognition[A].Proceedings of the International Conference on Image Processing[C].Barcelona:IEEE Computer Society,2003.855-858.
[4] 刘晓旻,章毓晋.基于Gabor直方图特征和MVBoost的人脸表情识别[J].计算机研究与发展,2007,44(7):1089-1096. Xiaomin L,Yujin Z.Facial expression recognition based on Gabor histogram feature and MVBoost[J].Journal of Computer Research and Development,2007,44(7):1089-1096.(in Chinese)
[5] Ahonen T,Hadid A,Pietikainen M.Face description with local binary patterns:Application to face recognition[J].Pattern Analysis and Machine Intelligence,IEEE Transactions on,2006,28(12):2037-2041.
[6] Cheon Y,Kim D.Natural facial expression recognition using differential-AAM and manifold learning[J].Pattern Recognition,2009,42(7):1340-1350.
[7] Sánchez A,Ruiz J V,Moreno A B,et al.Differential optical flow applied to automatic facial expression recognition[J].Neurocomputing,2011,74(8):1272-1282.
[8] Koelstra S,Pantic M,Patras I.A dynamic texture-based approach to recognition of facial actions and their temporal models[J].Pattern Analysis and Machine Intelligence,IEEE Transactions on,2010,32(11):1940-1954.
[9] Wu T,Bartlett M S,Movellan J R.Facial expression recognition using gabor motion energy filters[A].Proceedings of the Computer Vision and Pattern Recognition Workshops[C].San Francisco,CA:IEEE Computer Society,2010.42-47.
[10] Tong Y,Liao W,Ji Q.Facial action unit recognition by exploiting their dynamic and semantic relationships[J].Pattern Analysis and Machine Intelligence,IEEE Transactions on,2007,29(10):1683-1699.
[11] Zhang Y,Ji Q.Active and dynamic information fusion for facial expression understanding from image sequences[J].Pattern Analysis and Machine Intelligence,IEEE Transactions on,2005,27(5):699-714.
[12] 李永强.基于动态贝叶斯网络的人面部运动识别方法研究[D].哈尔滨:哈尔滨工业大学,2013.
[13] Yang P,Liu Q,Metaxas D N.Exploring facial expressions with compositional features[A].Proceedings of the Computer Vision and Pattern Recognition[C].San Francisco,CA:IEEE Computer Society,2010.2638-2644.
[14] Valstar M F,Pantic M.Fully automatic recognition of the temporal phases of facial actions[J].Systems,Man,and Cybernetics,Part B:Cybernetics,IEEE Transactions on,2012,42(1):28-43.
[15] Young J D,Santos Jr E.Introduction to temporal bayesian networks[ED/OL].http://www.cs.indiana.edu/event/maics96/Proceedings/Young/maics-96.html,1996.
[16] Arroyo-Figueroa G,Suear L E.A temporal Bayesian network for diagnosis and prediction[A].Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence[C].Stockholm:Eprint Arxiv,1999.13-20.
[17] Shi Y,Huang Y,Minnen D,et al.Propagation networks for recognition of partially ordered sequential action[A].Proceedings of Conference on Computer Vision and Pattern Recognition[C].Atlanta,GA:IEEE Computer Society,2004.862-869.
[18] 涂传飞.考虑时态信息的本体知识的不确定性推理研究[D].武汉:华中科技大学,2013.
[19] Allen J F.Maintaining knowledge about temporal intervals[J].Communications of the ACM,1983,26(11):832-843.
[20] Lucey P,Cohn J F,Kanade T,et al.The extended Cohn-Kanade Dataset (CK+):A complete dataset for action unit and emotion-specified expression[A].Proceedings of Conference on Computer Vision and Pattern Recognition Workshops[C].San Francisco,CA:IEEE Computer Society,2010.94-101.
[21] Pantic M,Valstar M,Rademaker R,et al.Web-based database for facial expression analysis[A].Proceedings of Conference on the Multimedia and Expo[C].Netherlands:IEEE Computer Society,2005.317-321.
[22] 陈雄.基于序列特征的随机森林表情识别[D].成都:电子科技大学,2013.
[23] Zhong L,Liu Q,Yang P,et al.Learning active facial patches for expression analysis[A].Proceedings of Conference on Computer Vision and Pattern Recognition[C].Providence,RI:IEEE Computer Society,2012.2562-2569.
[24] Cheon Y,Kim D.A natural facial expression recognition using differential-AAM and k-NNS[A].Proceedings of International Symposium on Multimedia[C].Berkeley,CA:IEEE Computer Society,2008.220-227.
[25] Wang L,Qiao Y,Tang X.Motionlets:Mid-level 3D parts for human motion recognition[A].Proceedings of Conference on Computer Vision and Pattern Recognition[C].Portland,OR:IEEE Computer Society,2013.2674-2681.
[26] Zhao G,Pietikainen M.Dynamic texture recognition using local binary patterns with an application to facial expressions[J].Pattern Analysis and Machine Intelligence,IEEE Transactions on,2007,29(6):915-928.
[27] Klaser A,Marszalek M.A spatio-temporal descriptor based on 3d-gradients[A].Proceedings of the the 19th British Machine Vision Conference[C].Leeds:British Machine Vision Association,2008.1-10.

结合运动时序性的人脸表情识别方法

Facial Expression Recognition Using Temporal Relations Among Facial Movements

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	吕阳, 钱斌, 胡蓉, 张梓琪. 增强人工蜂群算法求解半导体最终测试调度问题[J]. 电子学报, 2021, 49(9): 1708-1715.
[2]	李海, 尚金雷, 孙婷逸, 冯青, 庄子波. 一种基于离散属性BNT的双偏振气象雷达降水粒子分类方法[J]. 电子学报, 2021, 49(3): 619-624.
[3]	谭翔元, 高晓光, 贺楚超. 基于马尔科夫毯约束的最优贝叶斯网络结构学习算法[J]. 电子学报, 2019, 47(9): 1898-1904.
[4]	李宏菲, 李庆, 周莉. 基于多视觉描述子及音频特征的动态序列人脸表情识别[J]. 电子学报, 2019, 47(8): 1643-1653.
[5]	张瑞, 蒋晨之, 苏剑波. 基于稀疏特征挑选和概率线性判别分析的表情识别研究[J]. 电子学报, 2018, 46(7): 1710-1718.
[6]	李艳秋, 任福继, 胡敏. 动态模糊密度的多分类器融合算法[J]. 电子学报, 2018, 46(5): 1246-1252.
[7]	孙晓, 潘汀. 基于兴趣区域深度神经网络的静态面部表情识别[J]. 电子学报, 2017, 45(5): 1189-1197.
[8]	房丙午, 黄志球, 王勇, 李勇. 基于混合贝叶斯网络的混合系统安全性分析方法[J]. 电子学报, 2017, 45(12): 2896-2902.
[9]	胡敏, 江河, 王晓华, 许良凤, 黄晓音, 程轶红. 基于几何和纹理特征的表情层级分类方法[J]. 电子学报, 2017, 45(1): 164-172.
[10]	邸若海, 高晓光, 郭志高. 小数据集BN建模方法及其在威胁评估中的应用[J]. 电子学报, 2016, 44(6): 1504-1511.
[11]	房丙午, 黄志球, 李勇, 王勇. 基于贝叶斯网络的复杂系统动态故障树定量分析方法[J]. 电子学报, 2016, 44(5): 1234-1239.
[12]	刘钰, 韩峰, 陆希成, 王建国. 电子系统电磁脉冲易损性评估的分层贝叶斯网络模型[J]. 电子学报, 2016, 44(11): 2695-2703.
[13]	於俊, 汪增福, 李睿. 一种同步人脸运动跟踪与表情识别算法[J]. 电子学报, 2015, 43(2): 371-376.
[14]	蒋斌, 贾克斌. 一种用于表情识别的局部判别分量分析算法[J]. 电子学报, 2014, 42(1): 155-159.
[15]	朱明敏, 刘三阳, 杨有龙. 基于混合方式的贝叶斯网络等价类学习算法[J]. 电子学报, 2013, 41(1): 98-104.