基于人脸朝向的非穿戴自然人机交互

doi:10.3969/j.issn.0372-2112.2015.08.017

电子学报 ›› 2015, Vol. 43 ›› Issue (8): 1583-1588.DOI: 10.3969/j.issn.0372-2112.2015.08.017

基于人脸朝向的非穿戴自然人机交互

李雨龙¹, 管业鹏^1,2

1. 上海大学通信与信息工程学院, 上海 200444;
2. 新型显示技术及应用集成教育部重点实验室, 上海 200072

收稿日期:2014-02-13 修回日期:2014-07-29 出版日期:2015-08-25
- 通讯作者:
- 管业鹏
- 作者简介:
- 李雨龙 男,1988年生于湖南岳阳,上海大学通信与信息工程学院硕士研究生.主要研究方向为人机交互、智能信息感知与处理、计算机视觉与模式识别. E-mail:lylshu@163.com
- 基金资助:
- 国家自然科学基金 (No.11176016,No.60872117); 高等学校博士学科点专项科研基金 (No.20123108110014)

Human Face Orientation Based Non-Wearable and Natural Human-Computer Interaction

LI Yu-long¹, GUAN Ye-peng^1,2

1. School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China;
2. Key Laboratory of Advanced Displays and System Application, Ministry of Education, Shanghai 200072, China

Received:2014-02-13 Revised:2014-07-29 Online:2015-08-25 Published:2015-08-25
- Supported by:
- National Natural Science Foundation of China (No.11176016, No.60872117); Research Fund for the Doctoral Program of Higher Education of China (No.20123108110014)

摘要/Abstract

摘要：

提出了一种新颖的基于人脸朝向的非穿戴人机交互新方法.采用主动形状模型确定人脸面部轮廓特征点,克服光照和人脸姿态变化影响,在简化冗余信息的同时,降低计算复杂度.基于最大三角化划分面部轮廓特征,在此基础上,基于Kinect摄像机将构建的二维人脸模型映射至三维空间,动态求解面部朝向的法向量,通过图像帧间的均值滤波确定空间交互目标.用户无需佩带任何标记,且其活动不受约束,满足人机交互舒适性、多元性要求,实现非穿戴自然人机交互.通过实验对比,验证了本文方法有效、可行.

关键词: 人机交互, 人脸朝向, 非穿戴, 面部几何模型

Abstract:

A novel non-wearable human-computer interaction (HCI) method has been proposed based on facial orientation.An active shape model is employed to locate some feature points in the facial contour which can be able to overcome the effects of illumination and facial posture variation.Some redundant information can be simplified.Moreover, HCI can be made in a real time.After a 2D facial geometric model being built based on the maximal triangulation of facial contour features, a calibrated Kinect sensor is used to map the geometric model into 3D coordinate.In order to determine the interacting target, the normal direction of facial orientation is calculated dynamically by a temporal median filter.Non-wearable and natural HCI modal can be realized in which the user can move freely without wearing any equipment or auxiliary markers.Besides, the need of comfort and diversity can be met simultaneously during HCI.Experiment results indicate that the developed approach is efficient and can be used to natural non-wearable HCI.

Key words: human-computer interaction, facial orientation, non-wearable, facial geometric model

中图分类号:

TP39

李雨龙, 管业鹏. 基于人脸朝向的非穿戴自然人机交互[J]. 电子学报, 2015, 43(8): 1583-1588.

LI Yu-long, GUAN Ye-peng . Human Face Orientation Based Non-Wearable and Natural Human-Computer Interaction[J]. Acta Electronica Sinica, 2015, 43(8): 1583-1588.

参考文献

[1] 管业鹏.基于多模态视觉特征的自然人机交互[J].电子学报,2013,41(11):2223-2229. Guan Y P.Multimodal visual features based natural human-computer interaction[J].Acta Electronica Sinica,2013,41(11):2223-2229.(in Chinese)
[2] Mehrabian A.Communication without words[J].Psychology Today,1968,2(4):53-56.
[3] Van den Bergh M,Van Gool L.Combining RGB and ToF cameras for real-time 3D hand gesture interaction[A].Proceedings of IEEE Workshop on Applications of Computer Vision[C].Kona,United States:IEEE Computer Society,2011.66-72.
[4] 管业鹏.复杂人机交互场景下的指势用户对象识别[J].电子学报,2014,42(11):2135-2141. Guan Y P.Pointing user recognition in human-computer interaction with cluttered scene[J].Acta Electronica Sinica,2014,42(11):2135-2141.(in Chinese)
[5] Enfield N J,Kita S,Ruiter de J P.Primary and secondary pragmatic functions of pointing gestures[J].Journal of Pragmatics,2007,39(10):1722-1741.
[6] Colombo C,Bimbo A D,A.Valli.Visual capture and understanding of hand pointing actions in 3-D environment[J].IEEE Transactions on Systems,Man and Cybernetics-Part B:Cybernetics,2003,33(4):677-686.
[7] Van den Bergh M,Carton D,Nijs R D,et al.Real-time 3D hand gesture interaction with a robot for understanding directions from humans[A].Proceedings of IEEE International Workshop on RO-MAN[C].Atlanta,United States:IEEE,2011.357-362.
[8] Pan J,Guan Y P.Human-computer interaction using pointing gesture based on an adaptive virtual touch screen[J].International Journal of Signal Processing,Image Processing and Pattern Recognition,2013,6(4):81-92.
[9] 程成,杜菁菁,蓝飞翔.眼动交互的实时线性算法构造和实现[J].电子学报,2009,37(4A):12-15. Cheng C,Du J J,Lan F X.Research and realization on real-time linear prediction algorithm for desktop eye-tracking[J].Acta Electronica Sinica,2009,37(4A):12-15.(in Chinese)
[10] Sugano Y,Matsushita Y,Sato Y.Appearance-based gaze estimation using visual saliency[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(2):329-341.
[11] Tu Y J,Kao C C,Lin H Y.Human computer interaction using face and gesture recognition[A].Proceedings of IEEE International Conference on Signal and Information Processing[C].Kaohsiung,Taiwan:IEEE Computer Society,2013.1-8.
[12] Agrawal A,Raj R,Porwal S.Vision-based multimodal human-computer interaction using hand and head gestures[A].Proceedings of IEEE International Conference on Information and Communication Technologies[C].Thuckalay,India:IEEE Computer Society,2013.1288-1292.
[13] Carrino S,Péclat A,Mugellini E,Abou Khaled O,Ingold R.Humans and smart environments:a novel multimodal interaction approach[A].Proceedings of ACM International Conference on Multimodal Interaction[C].Alicante,Spain:Association for Computing Machinery,2011.105-112.
[14] Beymer D J.Face recognition under varying pose[A].Proceedings of IEEE Conference on Computer Vision and Pattern Recognition[C].Seattle,United States:IEEE,1994.756-761.
[15] Zhu Y,Fujimura K.Head pose estimation for driver monitoring[A].Proceedings of IEEE Conference on Intelligent Vehicles Symposium[C].Parma,Italy:IEEE,2004.501-506.
[16] Huang K S,Trivedi M M.Robust real-time detection,tracking and pose estimation of faces in video streams[A].Proceedings of IEEE Conference on Pattern Recognition[C].Cambridge,United Kingdom:IEEE,2004.3:965-968.
[17] Zhang Z Q,Hu Y X,Liu M,Huang T.Head pose estimation in seminar room using multi view face detectors[A].Proceedings of International Conference on Multimodal Technologies for Perception of Humans[C].Southhampton,United Kingdom:Springer Verlag,2007.299-304.
[18] Yang J L,Liang W,Jia Y D.Face pose estimation with combined 2D and 3D HOG features[A].Proceedings of IEEE Conference on Pattern Recognition[C].Tsukuba,Japan:IEEE,2012.2492-2495.
[19] Heo J,Savvides M.Generic 3D face pose estimation using facial shapes[A].Proceedings of International Conference on Biometrics[C].Washington DC,United States:IEEE Computer Society,2011.1-8.
[20] Ma Y,Konishi Y,Kinoshita K,Lao S,Kawade M.Sparse Bayesian regression for head pose estimation[A].Proceedings of International Conference on Pattern Recognition[C].Hong Kong,China:IEEE,2006.3:507-510.
[21] Ma B,Shan S,Chen X,Gao W.Head yaw estimation from asymmetry of facial appearance[J].IEEE Transactions on Systems,Man and Cybernetics-Part B,2008,38(6):1501-1512.
[22] Cootes T F,Edwards G J,Taylor C J.Active appearance models[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2001,23(6):681-685.
[23] Cootes T F,Taylor C J,Cooper D H,Graham J.Active shape models-their training and application[J].Computer Vision and Image Understanding,1995,61(1):38-59.
[24] Kittipanya-ngam P,Cootes T F.The effect of texture representations on AAM performance[A].Proceedings of IEEE Conference on Pattern Recognition[C].Hong Kong,China:IEEE,2006.2:328-331.
[25] Viola P,Jones M.Robust real-time face detection[J].International Journal of Computer Vision,2004,57(2):137-154.
[26] Daniel H,Kannala J,Heikkil J.Joint depth and color camera calibration with distortion correction[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(10):2058-2064.
[27] Qi F,Han J Y,Wang P J,Shi G M,Li F.Structure guided fusion for depth map inpainting[J].Pattern Recognition Letters,2013,34(1):70-76.

基于人脸朝向的非穿戴自然人机交互

Human Face Orientation Based Non-Wearable and Natural Human-Computer Interaction

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 14

编辑推荐

Metrics

[1]	谢佳龙, 张波涛, 吕强. 一种基于双流融合3D卷积神经网络的动态头势识别方法[J]. 电子学报, 2021, 49(7): 1363-1369.
[2]	杨珂, 王敬宇, 戚琦, 孙海峰, 王晶, 廖建新. LSCN:一种用于动作识别的长短时序关注网络[J]. 电子学报, 2020, 48(3): 503-509.
[3]	冯志全, 乔宇, 冯仕昌, 周晓燕, 徐涛, 杨晓晖, 范雪, 毕建平. 多种手势对应同一语义的柔性映射交互算法的研究[J]. 电子学报, 2019, 47(8): 1612-1617.
[4]	王勇, 王沙沙, 田增山, 周牧, 吴金君. 基于FMCW雷达的双流融合神经网络手势识别方法[J]. 电子学报, 2019, 47(7): 1408-1415.
[5]	管业鹏. 复杂人机交互场景下的指势用户对象识别[J]. 电子学报, 2014, 42(11): 2135-2141.
[6]	管业鹏. 基于多模态视觉特征的自然人机交互[J]. 电子学报, 2013, 41(11): 2223-2229.
[7]	武汇岳, 王建民, 戴国忠. 基于小样本学习的3D动态视觉手势个性化交互方法[J]. 电子学报, 2013, 41(11): 2230-2236.
[8]	陈明炫;邓昌智;任磊;田丰;戴国忠. 一种基于模型驱动的笔式界面开发框架研究[J]. 电子学报, 2011, 39(2): 268-274.
[9]	程成, 杜菁菁, 蓝飞翔. 眼动交互的实时线性算法构造和实现[J]. 电子学报, 2009, 37(S1): 12-15.
[10]	黄莹;王志良;涂序彦;戚颖. 一种可适应自然头动的视线追踪系统的研制及应用[J]. 电子学报, 2009, 37(4): 764-770.
[11]	金连文, 徐睿, 杨端端, 镇立新, 黄建成. 手指书写:一种虚拟文字识别人机交互新方法[J]. 电子学报, 2007, 35(3): 396-401.
[12]	徐一华, 李善青, 贾云得. 一种基于视觉的手指屏幕交互方法[J]. 电子学报, 2007, 35(11): 2236-2240.
[13]	袁保宗, 阮秋琦, 王延江, 刘汝杰, 唐晓芳. 新一代(第四代)人机交互的概念框架特征及关键技术[J]. 电子学报, 2003, 31(S1): 1945-1954.
[14]	任海兵;祝远新;徐光;林学;张哓平. 基于视觉手势识别的研究—综述[J]. 电子学报, 2000, 28(2): 118-121.