多种手势对应同一语义的柔性映射交互算法的研究

doi:10.3969/j.issn.0372-2112.2019.08.002

电子学报 ›› 2019, Vol. 47 ›› Issue (8): 1612-1617.DOI: 10.3969/j.issn.0372-2112.2019.08.002

多种手势对应同一语义的柔性映射交互算法的研究

冯志全¹, 乔宇¹, 冯仕昌², 周晓燕¹, 徐涛¹, 杨晓晖¹, 范雪¹, 毕建平³

1. 济南大学信息科学与工程学院, 山东济南 250022;
2. 厦门大学信息学院智能科学与技术系, 福建厦门 361005;
3. 山东中医药大学附属医院, 山东济南 250011

收稿日期:2018-03-03 修回日期:2019-02-28 出版日期:2019-08-25
- 通讯作者:
- 冯志全
- 作者简介:
- 乔宇男,1993年出生于河北辛集人.济南大学硕士学位.主要从事人机交互相关技术的研究.
- 基金资助:
- 国家重点研发计划 (No.2018YFB1004901）; 国家自然科学基金 (No.61472163，No.61603151）; 山东省重点研发计划 (No.2017GGX10146）

Research on Flexible Mapping Algorithm of Multi-gestures to One Semantic

FENG Zhi-quan¹, QIAO Yu¹, FENG Shi-chang², ZHOU Xiao-yan¹, XU Tao¹, YANG Xiao-hui¹, FAN Xue¹, BI Jian-ping³

1. College of Information Science and Engineering, University of Jinan, Jinan, Shandong 250022, China;
2. School of Information Science and Engineering, Xiamen University, Xiamen, Fujian 361005, China;
3. Affiliated Hospital of Shandong Universityof Traditional Chinese Medicine, Jinan, Shandong 250011, China

Received:2018-03-03 Revised:2019-02-28 Online:2019-08-25 Published:2019-08-25

摘要/Abstract

摘要： 针对智能交互界面中手势识别错误导致交互界面变化错误和手势不识别两个基本问题，本文设计并实现了基于手势交互的智能教学界面，该系统可以通过获取教师的手势信息与教师进行交互.主要创新点在于提出了多种手势对应同一语义的柔性映射交互算法.本文选取了14种自然交互手势，分析了对应同一语义的多种手势之间的共同特征.实验结果显示，该算法能够有效降低用户负荷.该算法已经用于一个基于手势交互的智能教学系统界面中.

关键词: 交互模型, 自然用户界面, 认知负担, 基于手势的交互, 人机交互

Abstract: Aiming at the two basic problems,the error of interface interaction caused by hand gesture recognition errors and gesture non-recognition error,the intelligent teaching interface based on gesture interaction is designed and implemented.And the system can interact with the teacher by obtaining the gesture information of the teacher.The main innovation of this paper is to propose a flexible mapping interaction algorithm with multiple gestures mapping to the one semantic.The paper selects fourteen kinds of natural interactive gesture,and analyzes the common characteristics of various gestures mapping to the one semantic.The experimental results show that the proposed algorithm can effectively reduce the users' burden.The algorithm has been used in an intelligent teaching system interface based on gesture interaction.

Key words: interactive mode, natural user interface, cognitive load, gesture interaction, human computer interaction

中图分类号:

TP391

冯志全, 乔宇, 冯仕昌, 周晓燕, 徐涛, 杨晓晖, 范雪, 毕建平. 多种手势对应同一语义的柔性映射交互算法的研究[J]. 电子学报, 2019, 47(8): 1612-1617.

FENG Zhi-quan, QIAO Yu, FENG Shi-chang, ZHOU Xiao-yan, XU Tao, YANG Xiao-hui, FAN Xue, BI Jian-ping. Research on Flexible Mapping Algorithm of Multi-gestures to One Semantic[J]. Acta Electronica Sinica, 2019, 47(8): 1612-1617.

参考文献

[1] Yin X,Xie M.Finger identification and hand posture recognition for human-robot interaction[J].Image and Vision Computing,2007,25(8):1291-1300.
[2] Nguyen,Viet T.Enhancing Touchless Interaction with the Leap Motion Uusing a Haptic Glove[D].Joensuu,Kuopio,Finland:University of Eastern Finland,2014.1-67.
[3] Riener A,Ferscha A,Bachmair F,et al.Standardization of the in-car gesture interaction space[A].Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications[C].New York,USA:ACM,2013.14-21.
[4] 齐静,徐坤,丁希仑.机器人视觉手势交互技术研究进展[J].机器人,2017,39(4):565-584.
[5] Ahn J,Kim K.Investigating smart TV gesture interaction based on gesture types and styles[J].Journal of the Ergonomics Society of Korea,2017,36(2):109-121.
[6] 徐治鹏,冯志全,刘慧,等.面向智能电视的隐式手势交互建模与算法[J].计算机辅助设计与图形学学报,2017,29(2):344-353. Xu Z,Feng Z,Liu H,et al.Implicitgesture interaction modeling and algorithm for smart TV[J].Journal of Computer-Aided Design & Computer Graphics,2017,29(2):344-353.(in Chinese)
[7] Cui J,Kuijper A,Sourin A.Exploration of natural free-hand interaction for shape modeling using leap motion controller[A].Proceedings of the International Conference on Cyberworlds(CW)[C].Chongqing,China:IEEE,2016.41-48.
[8] 张玉军,孟晓军,王刚.基于手势交互的三维电子沙盘系统设计与实现[J].指挥控制与仿真,2016,38(2):110-114.
[9] Carter M,Newn J,Velloso E,et al.Remote Gaze and Gesture Tracking on the microsoft kinect:investigating the role of feedback[A].Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction[C].New York,US:ACM,2015.167-176.
[10] Ren Z,Yuan J,Meng J,et al.Robust part-based hand gesture recognition using Kinect sensor[J].IEEE Transactions on Multimedia,2013,15(5):1110-1120.
[11] Camastra F,De Felice D.LVQ-based hand gesture recognition using a data glove[A].Neural Nets and Surroundings[M].Berlin Heidelberg:Springer,2013,159-168.
[12] 张莹莹,郭星.基于Kinect动态手势识别算法的研究与实现[J].计算机技术与发展,2017,27(12):11-15.
[13] Hu Z F,Wang Y K,Luo Y,et al.An Improved camshift-based particle filter algorithm for real-time hand gesture tracking[J].Sensors & Transducers,2014,177(8):307-312.
[14] Sanchez-Riera J,Hsiao Y S,Lim T,et al.A robust tracking algorithm for 3d hand gesture with rapid hand motion through deep learning[A].Proceedings of the 2014 IEEE International Conference on Multimedia and Expo Workshops(ICMEW)[C].Cheng Du,China:IEEE,2014.1-6.
[15] Ju Z,Wang Y,Zeng W,et al.A modified EM algorithm for hand gesture segmentation in RGB-D data[A].Proceedings of the 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)[C].Beijing,China:IEEE,2014.1736-1742.
[16] Huang L P,Wu Q X,Zhang G R,et al.Gesture segmentation using spiking neural networks inspired by the visual system[A].Proceedings of 2015 International Conference on Artificial Intelligence and Industrial Engineering[C].Thailand,Phuket:Atlantis Press,2015.184-187.
[17] Zhang Q,Lu J,Zhang M,et al.Hand gesture segmentation based on mixed skin-color model and FCM algorithm[J].Journal of Information &Computational Science,2015,12(9):3527-3536.
[18] Song Y,Davis R.Continuous body and hand gesture recognition for natural human-computer interaction[A].Proceedings of the 24th International Conference on Artificial Intelligence[C].Buenos Aires,Argentina:AAAI Press,2015.4212-4216.
[19] 刘杰,黄进,韩冬奇,等.模板匹配的三维手势识别算法[J].计算机辅助设计与图形学报,2016,28(8):1365-1372. Liu J,Huang J,Han D,et al.Templatematching algorithm for 3D gesture recognition[J].Journal of Computer-Aided Design & Computer Graphics,2016,28(8):1365-1372.(in Chinese).
[20] Wang K R,Xiao B J,Xia J Y,et al.A real-time vision-based hand gesture interaction system for virtual EAST[J].Fusion Engineering and Design,2016,112:829-834.
[21] Qian K,Niu J,Yang H.Developing a gesture based remote human-robot interaction system using Kinect[J].International Journal of Smart Home,2013,7(4):203-208.
[22] 李文瀚,郑新,金宁,等.基于手势识别的智能机器人系统设计[J].中国科技信息,2018,575(2):76-77.
[23] Kumar P,Verma J,Prasad S.Hand data glove:a wearable real-time device for human-computer interaction[J].International Journal of Advanced Science and Technology,2012,43:15-26.
[24] 李东洁,李洋洋,杨柳.GL-RBF优化的数据手套手势识别算法[J].哈尔滨理工大学学报,2017,22(4):7-12.
[25] Yang X W,Feng Z Q,Huang Z Z,et al.Gesture recognition based on combining main direction of gesture and hausdorff-like distance[J].Journal of Computer-Aided Design & Computer Graphocs.2016,28(1):75-81.
[26] Zhang Y,Wu S H,Luo Y.Applications and recognition of gesture trajectory using HMM[J].Semiconductor Optoelectronics,2015,36(4):650-656.
[27] Xiaoyan Z,Zhiquan F,Yu Q,et al.The Man-machine finger-guessing game based on cooperation mechanism[J].Theoretical Computer Science,2017,6:92-109.

多种手势对应同一语义的柔性映射交互算法的研究

Research on Flexible Mapping Algorithm of Multi-gestures to One Semantic

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 14

编辑推荐

Metrics

[1]	谢佳龙, 张波涛, 吕强. 一种基于双流融合3D卷积神经网络的动态头势识别方法[J]. 电子学报, 2021, 49(7): 1363-1369.
[2]	杨珂, 王敬宇, 戚琦, 孙海峰, 王晶, 廖建新. LSCN:一种用于动作识别的长短时序关注网络[J]. 电子学报, 2020, 48(3): 503-509.
[3]	王勇, 王沙沙, 田增山, 周牧, 吴金君. 基于FMCW雷达的双流融合神经网络手势识别方法[J]. 电子学报, 2019, 47(7): 1408-1415.
[4]	李雨龙, 管业鹏. 基于人脸朝向的非穿戴自然人机交互[J]. 电子学报, 2015, 43(8): 1583-1588.
[5]	管业鹏. 复杂人机交互场景下的指势用户对象识别[J]. 电子学报, 2014, 42(11): 2135-2141.
[6]	管业鹏. 基于多模态视觉特征的自然人机交互[J]. 电子学报, 2013, 41(11): 2223-2229.
[7]	武汇岳, 王建民, 戴国忠. 基于小样本学习的3D动态视觉手势个性化交互方法[J]. 电子学报, 2013, 41(11): 2230-2236.
[8]	陈明炫;邓昌智;任磊;田丰;戴国忠. 一种基于模型驱动的笔式界面开发框架研究[J]. 电子学报, 2011, 39(2): 268-274.
[9]	程成, 杜菁菁, 蓝飞翔. 眼动交互的实时线性算法构造和实现[J]. 电子学报, 2009, 37(S1): 12-15.
[10]	黄莹;王志良;涂序彦;戚颖. 一种可适应自然头动的视线追踪系统的研制及应用[J]. 电子学报, 2009, 37(4): 764-770.
[11]	金连文, 徐睿, 杨端端, 镇立新, 黄建成. 手指书写:一种虚拟文字识别人机交互新方法[J]. 电子学报, 2007, 35(3): 396-401.
[12]	徐一华, 李善青, 贾云得. 一种基于视觉的手指屏幕交互方法[J]. 电子学报, 2007, 35(11): 2236-2240.
[13]	袁保宗, 阮秋琦, 王延江, 刘汝杰, 唐晓芳. 新一代(第四代)人机交互的概念框架特征及关键技术[J]. 电子学报, 2003, 31(S1): 1945-1954.
[14]	任海兵;祝远新;徐光;林学;张哓平. 基于视觉手势识别的研究—综述[J]. 电子学报, 2000, 28(2): 118-121.