A Simultaneous Facial Motion Tracking and Expression Recognition Algorithm
YU Jun1,2, WANG Zeng-fu1,2,3, LI Rui2,3
1. National Laboratory of Speech and Language Information Processing, University of Science and Technology of China, Hefei, Anhui 230027, China;
2. Department of Automation, University of Science and Technology of China, Hefei, Anhui 230027, China;
3. Institute of Intelligent Machines, Chinese Academy of Sciences, Hefei, Anhui 230031, China
In view of facial expression recognition from monocular video with dynamic background,a real-time system was proposed based on the algorithm in which facial motion is tracked and facial expression is recognized simultaneously.Firstly,online appearance model and cylinder head model were combined to track 3D facial motion from video in framework of particle filtering;secondly,the static knowledge of facial expression was extracted through facial expression anatomy;thirdly,the dynamic knowledge of facial expression was extracted through manifold learning;fourthly,facial expression was retrieved by fusing the static knowledge and dynamic knowledge during facial motion tracking process.The experiments results confirmed the advantage on facial expression recognition even in the presence of significant head pose and facial expression variations of this system.
於俊, 汪增福, 李睿. 一种同步人脸运动跟踪与表情识别算法[J]. 电子学报, 2015, 43(2): 371-376.
YU Jun, WANG Zeng-fu, LI Rui. A Simultaneous Facial Motion Tracking and Expression Recognition Algorithm. Chinese Journal of Electronics, 2015, 43(2): 371-376.
[1] P Ekman,W V Friesen.Manual for the Facial Action Coding System[M].Palo Alto,CA,USA:Psychologists Press,1978.
[2] J Sung,T Kanade,D Kim.Pose robust face tracking by combining active appearance models and cylinder head models[J].International Journal of Computer Vision,2008,80(2):260-274.
[3] M Bartlett,G Littlewort,C Lainscsek.Machine learning methods for fully automatic recognition of facial expressions and facial actions[A].ICSMC[C].Vancouver,BC:IEEE Xplore,2004.1574-1579.
[4] Wang Y,Ai H,Wu B,Huang C.Real time facial expression recognition with Adaboost[A].ICPR[C].Vancouver,BC:IEEE Xplore,2004.1683-1688.
[5] Tang F Q,Deng B,Facial expression recognition using AAM and local facial feature[A].ICNC[C].Vancouver,BC:IEEE Xplore,2007.632-635.
[6] Hammal Z,Couvreur L,Caplier A,Facial expression classification:An approach based on the fusion of facial deformations using the transferable belief model[J].IJAR,2007,46(3):542-567.
[7] Kotsia I,Zafeiriou S,Pitas L,Texture and shape information fusion for facial expression and facial action unit recognition[J].PR,2008,41(3):833-851.
[8] Cohen I,Sebe N,Garg A,Chen L,Huang T S.Facial expression recognition from video sequences:Temporal and static modeling[J].CVIU,2003,91(1-2):160-187.
[9] Zhang Y,Ji Q.Active and dynamic information fusion for facial expression understanding from image sequences[J].TPAMI,2005,27(5):699-714.
[10] Zhu Z,Ji Q,Robust pose invariant facial feature detection and tracking in real time[A].ICPR[C].Vancouver,BC:IEEE Xplore,2006.1092-1095.
[11] Tong Y,Wang Y,Zhu Z W,Facial feature tracking using a multi-state hierarchical shape model under varying face pose and facial expression[A].ICPR[C].Vancouver,BC:IEEE Xplore,2006.283-286.
[12] North B,Blake A,Isard M,Rittscher J.Learning and classification of complex dynamics[J].PAMI,2000,22(9):1016-1034.
[13] Zhou S,Krueger V,Chellappa R.Probabilistic recognition of human faces from video[J].CVIU,2003,91(1-2):214-245.
[14] Wei Zhang,et al.Real time feature based 3-D deformable face tracking[A].ECCV[C].Berlin,Heidelberg:Springer-Verlag,2008.720-732.
[15] M L Cascia,et al.Fast,reliable head tracking under varying illumination:An approach based on registration of texture mapped 3D models[J].TPAMI,2000,22(4):322-336.
[16] J Ahlberg.Model-based coding:Extraction,coding,and evaluation of face model parameters[D].Sweden:Linkøping University,2002.
[17] S Zhou,et al.Visual tracking and recognition using appearance-adaptive models in particle filters[J].TIP,2004,13(11):1491-1506.
[18] Lui Y M,et al.Adaptive appearance model and condensation algorithm for robust face tracking[J].TSMC Part A,2010,40(3):437-448.
[19] F Dornaika,F Davoine.Simultaneous facial action tracking and expression recognition in the presence of head motion[J].IJCV,2008,76(3):257-281.
[20] Xiao J,Moriyama T,Kanade T.Robust full-motion recovery of head by dynamic templates and registration techniques[J].IJIST,2003,13:85-94.
[21] Basu S,Essa I,Pentland A.Motion regularization for model-based head tracking[A].ICPR[C].Vancouver,BC:IEEE Xplore,1996.611-616.
[22] Malciu M,Preteux F.A robust model-based approach for 3D head tracking in video sequences[A].ICAFGR[C].Vancouver,BC:IEEE Xplore,2000.169-174.
[23] 於俊,汪增福.一种鲁棒高精度的人脸三维运动跟踪算法[J].计算机研究与发展,2014,51(4):802-812. Yu Jun,Wang Zeng-fu.A robust and high accurate 3D facial motion tracking algorithm[J].Journal of Computer Research and Development,2014,51(4):802-812.(in Chinese)
[24] J Ahlberg.CANDIDE3-an updated parameterized face[R].Technical Report LiTH-ISY-R-2326,Sweden:Department of Electrical Engineering,Linkøping University,2001.1-16.
[25] C Shan,S Gong,et al,Dynamic facial expression recognition using a bayesian temporal manifold model[A].British Machine Vision Conference[C].Vancouver,BC:IEEE Xplore,2006.2364-2369.
[26] K Lee,J Ho,M Yang,D Kriegman,Video-based face recognition using probabilistic appearance manifolds[A].CVPR[C].Vancouver,BC:IEEE Xplore,2003.1784-1789.
[27] Y Chang,C Hu,M Turk,Probabilistic expression analysis on manifolds[A].CVPR[C].Vancouver,BC:IEEE Xplore,2004.520-527.
[28] TKanade,J Cohn,Y Tian.Comprehensive database for facial expression analysis[A].ICAFGR[C].Vancouver,BC:IEEE Xplore,2000.46-53.
[29] M Muller.Information Retrieval for Music and Motion[M].Springer Berlin Heidelberg,2007.
2015, Vol. 43 
