基于FMCW雷达的双流融合神经网络手势识别方法

doi:10.3969/j.issn.0372-2112.2019.07.003

摘要
图/表
参考文献(0)
相关文章 (15)

全文: PDF (1342 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要针对传统光学摄像头和无线技术的手势识别方法受光照环境影响和空间纵向、横向特征不全的问题，该文提出一种基于调频连续波（Frequency Modulated Continuous Wave，FMCW）雷达信号的双流融合神经网络（Two-Stream Fusion Neural Network，TS-FNN）手势识别方法.首先，利用二维快速傅立叶变换（Fast Fourier Transform，FFT）求取中频信号的频谱，估计手势的距离和速度，并利用多重信号分类（Multiple Signal Classification，MUSIC）方法计算角度.其次，利用这三维参数在时间上的累积，将一个手势动作映射为32帧距离-速度矩阵图和角度时间图.最后，建立TS-FNN进行手势特征提取和特征融合.实验结果表明，TS-FNN方法与传统卷积神经网络相比，手势的平均识别准确率提升了约5%.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	王勇
	王沙沙
	田增山
	周牧
	吴金君

关键词 ：人机交互, 手势识别, FMCW雷达, 深度学习

Abstract：To deal with the problem of easily being affected by illumination environment of the traditional optical camera based hand gesture recognition method and the incomplete spatial and lateral characteristics of the wireless based hand gesture recognition method,this paper proposes a frequency modulated continuous wave (FMCW) radar signal based two-stream fusion neural network (TS-FNN) for hand gesture recognition.Firstly,the spectrum of the IF signal is obtained by two-dimensional Fast Fourier Transform (2D-FFT),the range and speed of the gesture are estimated,and the angle is calculated by the Multiple Signal Classification (MUSIC) method.Secondly,using the accumulation of three-dimensional parameters in time,a gesture action is mapped to a 32-frame range-speed matrix diagram and an angular-time map.Finally,TS-FNN is established for gesture feature extraction and classification.The experimental results show that compared with the existing methods,the TS-FNN method improves the average recognition accuracy by about 5%.

Key words： human-machine interaction gesture recognition FMCW radar deep learning

收稿日期: 2018-07-03 出版日期: 2019-04-09

ZTFLH:

TN961

基金资助:国家自然科学基金（No.61771083，No.61704015）；长江学者和创新团队发展计划（No.IRT1299）；重庆市科委重点实验室专项经费-重庆市基础科学与前沿技术研究项目（No.cstc2017jcyjAX0380，No.cstc2015jcyjBX0065）；重庆市高校优秀成果转化资助项目（No.KJZH17117）

作者简介: 王勇男,1987年9月出生,云南昭通人.重庆邮电大学讲师,IEEE会员.2010年、2012年和2018年分别获哈尔滨工业大学学士、硕士和博士学位.主要研究方向为无线通信、能效优化、深度学习等.E-mail:yongwang@cqupt.edu.cn;王沙沙 女,1992年9月出生,重庆沙坪坝人.2016年在重庆邮电大学获工学学士学位.现为重庆邮电大学硕士生,研究方向为手势识别技术.E-mail:1259657562@qq.com

引用本文:

王勇, 王沙沙, 田增山, 周牧, 吴金君. 基于FMCW雷达的双流融合神经网络手势识别方法[J]. 电子学报, 2019, 47(7): 1408-1415. WANG Yong, WANG Sha-sha, TIAN Zeng-shan, ZHOU Mu, WU Jin-jun. Two-Stream Fusion Neural Network Approach for Hand Gesture Recognition Based on FMCW Radar. Acta Electronica Sinica, 2019, 47(7): 1408-1415.

链接本文:

http://www.ejournal.org.cn/CN/10.3969/j.issn.0372-2112.2019.07.003 或 http://www.ejournal.org.cn/CN/Y2019/V47/I7/1408

[1] COELHO Y L,SALOMAO J M,KULITZ H R.Intelligent hand posture recognition system integrated to process control[J].IEEE Latin America Transactions,2017,15(6):1144-1153.
[2] 冯志全,杨学文,徐涛,等.结合手势二进制编码和类-Hausdorff距离的手势识别[J].电子学报,2017,45(9):2281-2291. FENG Zhi-quan,YANG Xue-wen,XU Tao,et al.Gesture recognition combining gesture binary coding and class-Hausdorff distance[J].Acta Electronica Sinica,2017,45(9):2281-2291.(in Chinese)
[3] PAI N S,HONG J H,CHEN P Y,et al.Application of design of image tracking by combining SURF and TLD and SVM-based posture recognition system in robbery pre-alert system[J].Multimedia Tools and Applications,2017,76(23):25321-25342.
[4] KHAN F,LEEM S K,CHO S H.Hand-based gesture recognition for vehicular applications using IR-UWB radar[J].Sensors,2017,17(4):833.
[5] ZHOU Z,CAO Z,PI Y.Dynamic gesture recognition with a terahertz radar based on range profile sequences and doppler signatures[J].Sensors,2017,18(1):10.
[6] WANG W,LIU A X,SHAHZAD M,et al.Device-free human activity recognition using commercial WiFi devices[J].IEEE Journal on Selected Areas in Communications,2017,35(5):1118-1131.
[7] LI Y,WANG X,LIU W,et al.Deep attention network for joint hand gesture localization and recognition using static RGB-D images[J].Information Sciences,2018,441:66-78.
[8] MOLCHANOV P,GUPTA S,KIM K,et al.Hand gesture recognition with 3D convolutional neural networks[A].Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops(CVPRW)[C].Boston:IEEE Computer Society,2015.1-7.
[9] ZHU G,ZHANG L,SHEN P,et al.Multimodal gesture recognition using 3D convolution and convolutional LSTM[J].IEEE Access,2017,99:1-1.
[10] TRAN D,BOURDEV L,FERGUS R,et al.Learning spatiotemporal features with 3D convolutional networks[A].Proceedings of the IEEE International Conference on Computer Vision[C].Washington:IEEE Computer Society,2015.4489-4497.
[11] WANG S,SONG J,LIEN J,et al.Interacting with Soli:exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum[A].Proceedings of the 29th Annual Symposium on User Interface Software and Technology[C].Tokyo:ACM,2016.851-860.
[12] LI G,ZHANG R,RITCHIE M,et al.Sparsity-based dynamic hand gesture recognition using micro-doppler signatures[A].Proceedings of the 2017 IEEE Radar Conference[C].Seattle:IEEE,2017.0928-0931.
[13] 郭跃,王宏迿,周陬.阵元间距对MUSIC算法的影响[J].电子学报,2007,35(9):1675-1679. GUO Yue,WANG Hong-xun,ZHOU Zou.The influence of element spacing on MUSIC algorithm[J].Acta Electronica Sinica,2007,35(9):1675-1679.(in Chinese)
[14] LIN J J,LI Y P,HSU W C,et al.Design of an FMCW radar baseband signal processing system for automotive application[J].SpringerPlus,2016,5(1):42.
[15] 王元恺,肖泽,许建中,等.一种改进的FMCW雷达线性调频序列波形[J].电子学报,2017,45(6):1288-1293. WANG Yuan-kai,XIAO Ze,XU Jian-zhong,et al.An improved FMCW radar linear frequency modulation sequence waveform[J].Acta Electronica Sinica,2017,45(6):1288-1293.(in Chinese)
[16] JARDAK S,AHMED S,ALOUINI M S.Low complexity moving target parameter estimation for MIMO radar using 2D-FFT[J].IEEE Transactions on Signal Processing,2017,65(18):4745-4755.
[17] PAN H,ZHANG F,SHI C,et al.High-precision frequency estimation for frequency modulated continuous wave laser ranging using the multiple signal classification method[J].Applied Optics,2017,56(24):6956-6961.
[18] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networksfor Large-Scale Image Recognition[OL].https://arxiv.org/pdf/1409.1556.pdf,2014.