电子学报 ›› 2022, Vol. 50 ›› Issue (4): 921-931.DOI: 10.12263/DZXB.20211069

• 机器学习交叉融合创新 • 上一篇    下一篇

基于时空图神经网络的手势识别

袁冠1,2, 邴睿1, 刘肖1, 代伟3, 张艳梅1, 蔡卓1   

  1. 1.矿山数字化教育部工程研究中心,江苏 徐州 221116
    2.中国矿业大学计算机科学与技术学院,江苏 徐州 221116
    3.中国矿业大学信息与控制工程学院,江苏 徐州 221116
  • 收稿日期:2021-08-11 修回日期:2021-12-12 出版日期:2022-04-25 发布日期:2022-04-25
  • 作者简介:袁 冠 男,1982年生,江苏睢宁人.现为中国矿业大学计算机科学与技术学院教授.主要研究方向为时空大数据技术以及计算智能.E-mail: yuanguan@cumt.edu.cn
    邴 睿 男,1994年生,甘肃兰州人. 现为中国矿业大学博士生.主要研究方向为图数据挖掘. E-mail: bingrui@cumt.edu.cn
    刘 肖 男,1994年生,江苏徐州人. 主要研究方向为模式识别与感知计算. E-mail: liuxiaocumt2018@163.com
  • 基金资助:
    国家自然科学基金(71774159);中国博士后科学基金(2021T140707)

Spatial-Temporal Graph Neural Network based Hand Gesture Recognition

YUAN Guan1,2, BING Rui1, LIU Xiao1, DAI Wei3, ZHANG Yan-mei1, CAI Zhuo1   

  1. 1.Digitization of Mine,Engineering Research Center of Ministry of Education,Xuzhou,Jiangsu 221116,China
    2.School of Computer Science and Technology,China University of Mining and Technology,Xuzhou,Jiangsu 221116,China
    3.School of Information and Control Engineering,China University of Mining and Technology,Xuzhou,Jiangsu 221116,China
  • Received:2021-08-11 Revised:2021-12-12 Online:2022-04-25 Published:2022-04-25

摘要:

随着感知计算以及传感器集成技术的发展,使用各种传感设备实时捕捉的手势运动数据,为人机交互提供了新的驱动力,并被广泛地应用于智能家居、远程医疗、虚拟现实等领域.由于手势动作具有时序性与空间连接性,因此在手势识别中需要考虑手势空间连接关系和手势长距离依赖特性.然而现有的手势识别方法忽略了上述两种特性,导致识别精度不高.本文提出了基于时空图神经网络的手势识别算法,该方法从传感器空间分布角度出发,基于传感器的空间位置信息,借助图神经网络(Graph Neural Networks,GNN)对手势数据之间的空间关联性进行表征,并引入门控循环单元(Gated Recurrent Unit,GRU)解决手势的时序性和长距离依赖问题,增强手势识别性能.在多种数据集上的实验结果证明本文方法可行且有效.

关键词: 手势识别, 多传感器融合, 时空图神经网络, 循环神经网络

Abstract:

With the development of perceptual computing and sensor integration technology, hand gesture motion data collected by various sensor devices provides a new data-driven way for human-computer interaction, and widely used in smart home, telemedicine, virtual reality and other fields. Due to hand gestures have temporality and spatial connectivity, it is necessary to consider spatial connection and long-distance dependence of hand gesture in gesture recognition. However, existing hand gesture recognition models ignore the aforementioned two problems, resulting in low recognition accuracy. Therefore, we propose a spatial-temporal graph neural network based hand gesture recognition model(STGNN-HGR). From the perspective of spatial distribution of sensors, based on the spatial location information of sensors, the model represents spatial correlation of hand gesture data with the help of graph neural networks(GNN), and introduces gated recurrent unit(GRU) to solve temporality and long-distance dependence in dynamic hand gestures, so as to enhance the performance of gesture recognition. The experimental results on a variety of datasets show that our model is feasible and effective.

Key words: hand gesture recognition, multi-sensor fusion, spatial-temporal graph neural network, recurrent neural network

中图分类号: