一种同步人脸运动跟踪与表情识别算法

doi:10.3969/j.issn.0372-2112.2015.02.025

电子学报 ›› 2015, Vol. 43 ›› Issue (2): 371-376.DOI: 10.3969/j.issn.0372-2112.2015.02.025

一种同步人脸运动跟踪与表情识别算法

於俊^1,2, 汪增福^1,2,3, 李睿^2,3

1. 中国科学技术大学语音及语言信息处理国家工程实验室, 安徽合肥 230027;
2. 中国科学技术大学自动化系, 安徽合肥 230027;
3. 中国科学院合肥智能机械研究所, 安徽合肥 230031

收稿日期:2013-03-11 修回日期:2014-03-31 出版日期:2015-02-25
- 作者简介:
- 於俊 男,1983年生于安徽滁州.中国科学技术大学语音及语言信息处理国家工程实验室副研究员.研究方向为人机情感接口,智能机器人. E-mail:harryjun@ustc.edu.cn;汪增福 男,1960年生于安徽合肥.教授,研究方向为计算机视听觉,智能机器人. E-mail:zfwang@ustc.edu.cn
- 基金资助:
- 国家自然科学基金 (No.61303150); 浙江大学CAD&CG国家重点实验室开放课题 (No.A1501); 中央高校基本科研业务费专项资金青年创新基金 (No.WK2100100020); 安徽省自主创新专项资金智能语音技术研发和产业化专项 (No.13Z02008)

A Simultaneous Facial Motion Tracking and Expression Recognition Algorithm

YU Jun^1,2, WANG Zeng-fu^1,2,3, LI Rui^2,3

1. National Laboratory of Speech and Language Information Processing, University of Science and Technology of China, Hefei, Anhui 230027, China;
2. Department of Automation, University of Science and Technology of China, Hefei, Anhui 230027, China;
3. Institute of Intelligent Machines, Chinese Academy of Sciences, Hefei, Anhui 230031, China

Received:2013-03-11 Revised:2014-03-31 Online:2015-02-25 Published:2015-02-25
- Supported by:
- National Natural Science Foundation of China (No.61303150); Open Project of State Key Laboratory of CAD&CG, Zhejiang University (No.A1501); Youth Innovative Fund of Fundamental Research Funds for the Central Universities (No.WK2100100020); Independent Innovation Special Fund for Intelligent Voice Technology Research and Development and Industrialization of Anhui Province (No.13Z02008)

摘要/Abstract

摘要：

针对单视频动态变化背景下的人脸表情识别问题,提出了一种同步人脸运动跟踪和表情识别算法,并在此基础上构建了一个实时系统.该系统达到了如下目标:首先在粒子滤波框架下结合在线外观模型和柱状几何模型进行人脸三维运动跟踪;接着基于生理知识来提取人脸表情的静态信息;然后基于流形学习来提取人脸表情的动态信息;最后在人脸运动跟踪过程中,结合人脸表情静态信息和动态信息来进行表情识别.实验结果表明,该系统在大姿态和丰富表情下具有较好的综合优势.

关键词: 人脸运动跟踪, 人脸表情识别, 流形学习, 粒子滤波

Abstract:

In view of facial expression recognition from monocular video with dynamic background,a real-time system was proposed based on the algorithm in which facial motion is tracked and facial expression is recognized simultaneously.Firstly,online appearance model and cylinder head model were combined to track 3D facial motion from video in framework of particle filtering;secondly,the static knowledge of facial expression was extracted through facial expression anatomy;thirdly,the dynamic knowledge of facial expression was extracted through manifold learning;fourthly,facial expression was retrieved by fusing the static knowledge and dynamic knowledge during facial motion tracking process.The experiments results confirmed the advantage on facial expression recognition even in the presence of significant head pose and facial expression variations of this system.

Key words: facial motion tracking, facial expression recognition, manifold learning, particle filter

中图分类号:

TB391.9

於俊, 汪增福, 李睿. 一种同步人脸运动跟踪与表情识别算法[J]. 电子学报, 2015, 43(2): 371-376.

YU Jun, WANG Zeng-fu, LI Rui . A Simultaneous Facial Motion Tracking and Expression Recognition Algorithm[J]. Acta Electronica Sinica, 2015, 43(2): 371-376.

参考文献

[1] P Ekman,W V Friesen.Manual for the Facial Action Coding System[M].Palo Alto,CA,USA:Psychologists Press,1978.
[2] J Sung,T Kanade,D Kim.Pose robust face tracking by combining active appearance models and cylinder head models[J].International Journal of Computer Vision,2008,80(2):260-274.
[3] M Bartlett,G Littlewort,C Lainscsek.Machine learning methods for fully automatic recognition of facial expressions and facial actions[A].ICSMC[C].Vancouver,BC:IEEE Xplore,2004.1574-1579.
[4] Wang Y,Ai H,Wu B,Huang C.Real time facial expression recognition with Adaboost[A].ICPR[C].Vancouver,BC:IEEE Xplore,2004.1683-1688.
[5] Tang F Q,Deng B,Facial expression recognition using AAM and local facial feature[A].ICNC[C].Vancouver,BC:IEEE Xplore,2007.632-635.
[6] Hammal Z,Couvreur L,Caplier A,Facial expression classification:An approach based on the fusion of facial deformations using the transferable belief model[J].IJAR,2007,46(3):542-567.
[7] Kotsia I,Zafeiriou S,Pitas L,Texture and shape information fusion for facial expression and facial action unit recognition[J].PR,2008,41(3):833-851.
[8] Cohen I,Sebe N,Garg A,Chen L,Huang T S.Facial expression recognition from video sequences:Temporal and static modeling[J].CVIU,2003,91(1-2):160-187.
[9] Zhang Y,Ji Q.Active and dynamic information fusion for facial expression understanding from image sequences[J].TPAMI,2005,27(5):699-714.
[10] Zhu Z,Ji Q,Robust pose invariant facial feature detection and tracking in real time[A].ICPR[C].Vancouver,BC:IEEE Xplore,2006.1092-1095.
[11] Tong Y,Wang Y,Zhu Z W,Facial feature tracking using a multi-state hierarchical shape model under varying face pose and facial expression[A].ICPR[C].Vancouver,BC:IEEE Xplore,2006.283-286.
[12] North B,Blake A,Isard M,Rittscher J.Learning and classification of complex dynamics[J].PAMI,2000,22(9):1016-1034.
[13] Zhou S,Krueger V,Chellappa R.Probabilistic recognition of human faces from video[J].CVIU,2003,91(1-2):214-245.
[14] Wei Zhang,et al.Real time feature based 3-D deformable face tracking[A].ECCV[C].Berlin,Heidelberg:Springer-Verlag,2008.720-732.
[15] M L Cascia,et al.Fast,reliable head tracking under varying illumination:An approach based on registration of texture mapped 3D models[J].TPAMI,2000,22(4):322-336.
[16] J Ahlberg.Model-based coding:Extraction,coding,and evaluation of face model parameters[D].Sweden:Linkøping University,2002.
[17] S Zhou,et al.Visual tracking and recognition using appearance-adaptive models in particle filters[J].TIP,2004,13(11):1491-1506.
[18] Lui Y M,et al.Adaptive appearance model and condensation algorithm for robust face tracking[J].TSMC Part A,2010,40(3):437-448.
[19] F Dornaika,F Davoine.Simultaneous facial action tracking and expression recognition in the presence of head motion[J].IJCV,2008,76(3):257-281.
[20] Xiao J,Moriyama T,Kanade T.Robust full-motion recovery of head by dynamic templates and registration techniques[J].IJIST,2003,13:85-94.
[21] Basu S,Essa I,Pentland A.Motion regularization for model-based head tracking[A].ICPR[C].Vancouver,BC:IEEE Xplore,1996.611-616.
[22] Malciu M,Preteux F.A robust model-based approach for 3D head tracking in video sequences[A].ICAFGR[C].Vancouver,BC:IEEE Xplore,2000.169-174.
[23] 於俊,汪增福.一种鲁棒高精度的人脸三维运动跟踪算法[J].计算机研究与发展,2014,51(4):802-812. Yu Jun,Wang Zeng-fu.A robust and high accurate 3D facial motion tracking algorithm[J].Journal of Computer Research and Development,2014,51(4):802-812.(in Chinese)
[24] J Ahlberg.CANDIDE3-an updated parameterized face[R].Technical Report LiTH-ISY-R-2326,Sweden:Department of Electrical Engineering,Linkøping University,2001.1-16.
[25] C Shan,S Gong,et al,Dynamic facial expression recognition using a bayesian temporal manifold model[A].British Machine Vision Conference[C].Vancouver,BC:IEEE Xplore,2006.2364-2369.
[26] K Lee,J Ho,M Yang,D Kriegman,Video-based face recognition using probabilistic appearance manifolds[A].CVPR[C].Vancouver,BC:IEEE Xplore,2003.1784-1789.
[27] Y Chang,C Hu,M Turk,Probabilistic expression analysis on manifolds[A].CVPR[C].Vancouver,BC:IEEE Xplore,2004.520-527.
[28] TKanade,J Cohn,Y Tian.Comprehensive database for facial expression analysis[A].ICAFGR[C].Vancouver,BC:IEEE Xplore,2000.46-53.
[29] M Muller.Information Retrieval for Music and Motion[M].Springer Berlin Heidelberg,2007.

一种同步人脸运动跟踪与表情识别算法

A Simultaneous Facial Motion Tracking and Expression Recognition Algorithm

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	奚畅, 蔡志明, 袁骏. 被动声纳粒子滤波检测前跟踪的检测门限设置方法[J]. 电子学报, 2022, 50(7): 1722-1734.
[2]	刘晓燕, 陈亮, 焦振航, 卢祥晨. 单基站DTMB信号约束的PDR运动参数修正[J]. 电子学报, 2022, 50(4): 860-868.
[3]	蒋林, 向超, 朱建阳, 刘奇. 加载语义似然估计的粒子滤波重定位[J]. 电子学报, 2021, 49(2): 306-314.
[4]	黄鸿, 王丽华, 石光耀. 面向高光谱遥感影像分类的监督多流形鉴别嵌入方法[J]. 电子学报, 2020, 48(6): 1099-1107.
[5]	夏楠, 王珏, 李博. 基于粒子滤波和交互多模型的移动定位方法[J]. 电子学报, 2019, 47(1): 197-203.
[6]	张瑞, 蒋晨之, 苏剑波. 基于稀疏特征挑选和概率线性判别分析的表情识别研究[J]. 电子学报, 2018, 46(7): 1710-1718.
[7]	陈志敏, 吴盘龙, 薄煜明, 田梦楚, 岳聪, 顾福飞. 基于自控蝙蝠算法智能优化粒子滤波的机动目标跟踪方法[J]. 电子学报, 2018, 46(4): 886-894.
[8]	范舜奕, 管桦, 侯志强, 余旺盛, 戴铂. 基于多表观特征子模型更新的鲁棒视觉跟踪[J]. 电子学报, 2018, 46(2): 440-446.
[9]	王延江, 李蕙, 齐玉娟. 一种受人脑三阶段记忆机制启发的鲁棒运动目标跟踪方法[J]. 电子学报, 2017, 45(9): 2065-2070.
[10]	卢锦, 王鑫, 章为川. 基于代价参考粒子滤波的存在概率检测算法[J]. 电子学报, 2017, 45(6): 1498-1505.
[11]	刘龙, 樊波阳, 刘金星, 杨乐超. 面向运动目标检测的粒子滤波视觉注意力模型[J]. 电子学报, 2016, 44(9): 2235-2241.
[12]	吴昊, 孙晓燕, 郭玉堂, 刘路路, 沈晶. 保持粒子多样性的非退化粒子滤波方法研究[J]. 电子学报, 2016, 44(7): 1734-1741.
[13]	薛模根, 朱虹, 袁广林. 在线鲁棒判别式字典学习视觉跟踪[J]. 电子学报, 2016, 44(4): 838-845.
[14]	王伟, 余玉揆, 郝燕玲. 一种新型混合并行粒子滤波频率估计方法[J]. 电子学报, 2016, 44(3): 740-746.
[15]	张宏欣, 周穗华, 冯士民. 基于径向基-Galerkin解的反馈粒子滤波器[J]. 电子学报, 2016, 44(1): 95-100.