空间上下文与时序特征融合的交警指挥手势识别技术

doi:10.3969/j.issn.0372-2112.2020.05.018

电子学报 ›› 2020, Vol. 48 ›› Issue (5): 966-974.DOI: 10.3969/j.issn.0372-2112.2020.05.018

空间上下文与时序特征融合的交警指挥手势识别技术

张丞, 何坚, 王伟东

北京工业大学信息学部, 北京 100124

收稿日期:2019-07-10 修回日期:2019-10-25 出版日期:2020-05-25 发布日期:2020-05-25
通讯作者: 何坚
作者简介:张丞男,1993年11月出生,北京人.分别于2016年和2019年在北京工业大学获得学士、硕士学位.现为北京工业大学博士研究生,主要研究方向为智能人机交互和模式识别.
基金资助:
国家自然科学基金（No.61602016）；北京市科技计划项目（No.D171100004017003）

Visual Recognition of Chinese Traffic Police Gestures Based on Spatial Context and Temporal Features

ZHANG Cheng, HE Jian, WANG Wei-dong

Faculty of Information, Beijing University of Technology, Beijing 100124, China

Received:2019-07-10 Revised:2019-10-25 Online:2020-05-25 Published:2020-05-25

摘要/Abstract

摘要： 针对无人驾驶汽车快速准确识别交警指挥手势的需求，本文在分析交警指挥手势的关节铰接特征基础上，建立基于关节点和骨架的交警指挥手势模型；其次，引入卷积姿势机（Convolutional Pose Machine，CPM）提取交警指挥手势的关键节点，进而提取交警指挥手势中骨架的相对长度及其与重力加速度的夹角作为空间上下文特征，并引入长短时记忆网络（Long Short Term Memory，LSTM）提取交警指挥手势的时序特征；最后，设计了融合空间上下文和时序特征的交警指挥手势识别机（Chinese Traffic Police Gesture Recognizer，CTPGR），创建了包含8种交警指挥手势、时长约2小时的交警指挥手势视频库对CTPGR进行训练验证，并通过实验将CTPGR与已有交警手势识别算法进行了对比分析.实验证明CTPGR可以快速准确地识别交警指挥手势，系统对复杂背景和动态交警指挥手势具有较强的适应能力.

关键词: 交警指挥手势, 手势识别, 卷积姿势机, 长短时记忆, 特征提取

Abstract: According to the need for driver assistance systems and intelligent vehicles to quickly and accurately identify traffic police command gestures,the articulated features of traffic police gesture is firstly analyzed,and a model based on the key points and skeletons of the police gesture is established.Secondly,the convolutional posture machine (CPM) is introduced to extract the key points of the traffic police gesture.Then the relative lengths of the gesture skeletons and the angles between each skeleton w.r.t.gravity are extracted as the spatial context features of the traffic police gesture.Meanwhile,long-term memory (LSTM) is introduced to extract the temporal features of traffic police gestures.Finally,the Chinese traffic police gesture recognizer (CTPGR) based on CPM and LSTM is designed,and a two-hour traffic police gesture video is recorded to train and verify the CTPGR.Experimental results show that the CTPGR is capable of recognizing traffic police gestures with high accuracy,and is fast enough for online gesture prediction.

Key words: traffic police gestures, gesture recognition, convolutional pose machine, long short term memory, feature extraction

中图分类号:

TP391.4

张丞, 何坚, 王伟东. 空间上下文与时序特征融合的交警指挥手势识别技术[J]. 电子学报, 2020, 48(5): 966-974.

ZHANG Cheng, HE Jian, WANG Wei-dong. Visual Recognition of Chinese Traffic Police Gestures Based on Spatial Context and Temporal Features[J]. Acta Electronica Sinica, 2020, 48(5): 966-974.

参考文献

[1] Wang B,Yuan T.Traffic police gesture recognition using accelerometer[A].IEEE SENSORS Conference[C].Lecce-Italy,2008.1080-1083.
[2] Yuan T,Wang B.Accelerometer-based Chinese traffic police gesture recognition system[J].Chinese Journal of Electronics,2010,19(2):270-274.
[3] 杜友田,陈峰,徐文立,等.基于视觉的人的运动识别综述[J].电子学报,2007,35(1):84-90. DU You-tian,CHEN Feng,XU Wen-li,et al.A survey on the vision-based human motion recognition[J].Acta Electronica Sinica,2007,35(1):84-90.(in Chinese)
[4] Cai Z,Guo F.Max-covering scheme for gesture recognition of Chinese traffic police[J].Pattern Analysis and Applications,2015,18(2):403-418.
[5] Guo F,Tang J,Cai Z.Automatic recognition of Chinese traffic police gesture based on max-covering scheme[J].Advances in Information Sciences and Service Sciences,2013,5(1):428.
[6] Eichner M,Ferrari V.Human pose co-estimation and applications[J].IEEE transactions on pattern analysis and machine intelligence,2012,34(11):2282-2288.
[7] Eichner M,Marin-Jimenez M,Zisserman A,et al.2d articulated human pose estimation and retrieval in (almost) unconstrained still images[J].International Journal of Computer Vision,2012,99(2):190-214.
[8] 管业鹏.复杂人机交互场景下的指势用户对象识别[J].电子学报,2014,42(11):2135-2141. GUAN Ye-peng.Pointing user recognition in human-computer interaction with cluttered scene[J].Acta Electronica Sinica,2014,42(11):2135-2141.(in Chinese)
[9] Kang H,Lee C W,Jung K.Recognition-based gesture spotting in video games[J].Pattern Recognition Letters,2004,25(15):1701-1714.
[10] 张友梅,常发亮,刘洪彬.基于3D人体骨架的动作识别[J].电子学报,2017,45(4):906-911. ZHANG You-mei,CHANG Fa-liang,LIU Hong-bin.Action recognition based on 3D skeleton[J].Acta Electronica Sinica,2017,45(4):906-911.(in Chinese)
[11] Zhang Z.Microsoft kinect sensor and its effect[J].IEEE Multimedia,2012,19(2):4-10.
[12] Guo F,Tang J,Wang X.Gesture recognition of traffic police based on static and dynamic descriptor fusion[J].Multimedia Tools and Applications,2017,76(6):8915-8936.
[13] Le Q K,Pham C H,Le T H.Road traffic control gesture recognition using depth images[J].IEIE Transactions on Smart Processing & Computing,2012,1(1):1-7.
[14] Zhou Z,Li S,Sun B.Extreme learning machine based hand posture recognition in color-depth image[A].Chinese Conference on Pattern Recognition[C].Springer,2014.276-285.
[15] 罗会兰,童康,孔繁胜.基于深度学习的视频中人体动作识别进展综述[J].电子学报,2019,47(5):1162-1173. LUO Hui-lan,TONG Kang,KONG Fan-sheng.The progress of human action recognition in videos based on deep learning:a review[J].Acta Electronica Sinica,2019,47(5):1162-1173.(in Chinese)
[16] Fragkiadaki K,Levine S,Felsen P,et al.Recurrent network models for human dynamics[A].Proceedings of the IEEE International Conference on Computer Vision[C].US:IEEE,2015.4346-4354.
[17] Wei S-E,Ramakrishna V,Kanade T,et al.Convolutional pose machines[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].US:IEEE,2016.4724-4732.
[18] Cao Z,Simon T,Wei S-E,et al.Realtime multi-person 2d pose estimation using part affinity fields[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].US:IEEE,2017.7291-7299.
[19] Ramakrishna V,Munoz D,Hebert M,et al.Pose machines:Articulated pose estimation via inference machines[A].European Conference on Computer Vision[C].Berlin:Springer,2014.33-47.
[20] Wu J,Zheng H,Zhao B,et al.Large-scale datasets for going deeper in image understanding[A].2019 IEEE International Conference on Multimedia and Expo (ICME)[C].US:IEEE,2019.1480-1485.
[21] Glorot X,Bengio Y.Understanding the difficulty of training deep feedforward neural networks[A].Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics[C].US:ACM,2010.249-256.
[22] Pigou L,Van Den Oord A,Dieleman S,et al.Beyond temporal pooling:Recurrence and temporal convolutions for gesture recognition in video[J].International Journal of Computer Vision,2018,126(2-4):430-439.
[23] He K,Zhang X,Ren S,et al.Deep residual learning for image recognition[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].US:IEEE,2016.770-778.
[24] Huang G,Liu Z,Van Der Maaten L,et al.Densely connected convolutional networks[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].US:IEEE,2017.4700-4708.
[25] Ji S,Xu W,Yang M,et al.3D convolutional neural networks for human action recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,35(1):221-231.
[26] Xingjian S H I,Chen Z,Wang H,et al.Convolutional LSTM network:A machine learning approach for precipitation nowcasting[A].Advances in Neural Information Processing Systems[C].CSDN,2015.802-810.
[27] Guo F,Cai Z,Tang J.Chinese traffic police gesture recognition in complex scene[A].IEEE 10th International Conference on Trust,Security and Privacy in Computing and Communications[C].US:IEEE,2011.1505-1511.

空间上下文与时序特征融合的交警指挥手势识别技术

Visual Recognition of Chinese Traffic Police Gestures Based on Spatial Context and Temporal Features

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	袁冠, 邴睿, 刘肖, 代伟, 张艳梅, 蔡卓. 基于时空图神经网络的手势识别[J]. 电子学报, 2022, 50(4): 921-931.
[2]	刘狄, 钱慧, 王中风. 基于LSTM特征提取的有限新息率畸变信号重构[J]. 电子学报, 2022, 50(1): 217-225.
[3]	冯西安, 寇思玮, 谭伟杰, 毕杨. 水声信号处理中的稀疏表示理论及应用[J]. 电子学报, 2021, 49(9): 1840-1851.
[4]	林金朝, 李必禄, 李国权, 黄正文, 庞宇. 基于EWT和结构特征提取的心电信号R波识别算法[J]. 电子学报, 2021, 49(6): 1217-1223.
[5]	徐元博, 江祥奎, 李龙. 高阶对称差分解析能量算子在微弱轴承故障信号提取中的应用[J]. 电子学报, 2021, 49(4): 750-759.
[6]	高琛, 张帆, 高彦钊. 利用数据稀疏性的LSTM加速器设计[J]. 电子学报, 2021, 49(2): 209-215.
[7]	汪宇玲, 陈立, 黎明, 钟国韵, 何月顺, 常玉祥, 宋伟宁. 基于迹变换和旋转增量调制特征的模糊人脸识别[J]. 电子学报, 2021, 49(12): 2437-2448.
[8]	鲍中新, 文成林, 马雪. 一种基于数据变化率的预处理及主元分析故障诊断方法[J]. 电子学报, 2021, 49(11): 2234-2240.
[9]	李鹏伟, 姜宇谦, 薛飞扬, 黄佳佳, 徐超. 一种基于深度学习的强对抗性Android恶意代码检测方法[J]. 电子学报, 2020, 48(8): 1502-1508.
[10]	黄鸿, 王丽华, 石光耀. 面向高光谱遥感影像分类的监督多流形鉴别嵌入方法[J]. 电子学报, 2020, 48(6): 1099-1107.
[11]	翟颖, 陈渤. 基于稳健变分自编码模型的雷达高分辨距离像目标识别算法[J]. 电子学报, 2020, 48(6): 1149-1155.
[12]	罗会兰, 陈鸿坤. 基于深度学习的目标检测研究综述[J]. 电子学报, 2020, 48(6): 1230-1239.
[13]	王传旭, 胡小悦, 孟唯佳, 闫春娟. 基于多流架构与长短时记忆网络的组群行为识别方法研究[J]. 电子学报, 2020, 48(4): 800-807.
[14]	郭小英, 李文书, 钱宇华, 白茹意, 贾春花. 可计算图像复杂度评价方法综述[J]. 电子学报, 2020, 48(4): 819-826.
[15]	刘颖, 刘红燕, 范九伦, 公衍超, 李莹华, 王富平, 卢津. 基于深度学习的小目标检测研究与应用综述[J]. 电子学报, 2020, 48(3): 590-601.