Real-Time Gesture Recognition Algorithm Based on LoRa Signals

HAN Chong; DU Hao; GUO Jian; SUN Li-juan

doi:10.12263/DZXB.20250096

您当前的位置：

首页 >

文章列表页 >

Real-Time Gesture Recognition Algorithm Based on LoRa Signals

PAPERS | 更新时间：2025-12-27

- Real-Time Gesture Recognition Algorithm Based on LoRa Signals
- ACTA ELECTRONICA SINICA Vol. 53, Issue 8, Pages: 2738-2749(2025)
- 作者机构：
  
  1.南京邮电大学计算机学院，江苏南京 210023
  2.江苏省物联网智能感知与计算重点实验室，江苏南京 210023
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62272242)
- DOI：10.12263/DZXB.20250096
  CLC： TP391;
- Received：06 February 2025，
  
  Accepted：26 May 2025，
  
  Published：25 August 2025
- 稿件说明：
移动端阅览
韩崇, 杜昊, 郭剑, 等. 基于LoRa信号的实时手势识别算法[J]. 电子学报, 2025, 53(08): 2738-2749.

HAN Chong, DU Hao, GUO Jian, et al. Real-Time Gesture Recognition Algorithm Based on LoRa Signals[J]. Acta Electronica Sinica, 2025, 53(08): 2738-2749.
韩崇, 杜昊, 郭剑, 等. 基于LoRa信号的实时手势识别算法[J]. 电子学报, 2025, 53(08): 2738-2749. DOI：10.12263/DZXB.20250096

HAN Chong, DU Hao, GUO Jian, et al. Real-Time Gesture Recognition Algorithm Based on LoRa Signals[J]. Acta Electronica Sinica, 2025, 53(08): 2738-2749. DOI：10.12263/DZXB.20250096

摘要

为了解决在遮挡环境下的实时手势识别问题，本文提出了一种基于远距离无线电（Long range Radio，LoRa）信号的实时手势识别算法.该算法利用LoRa信号频段较低、穿透性较好的特性，通过两根接收天线计算信号比值，并结合短时傅里叶变换（Short-Time Fourier Transform，STFT）得到包含手部运动特征的时频图.在此基础上，设计了Gesture Encoder编码器进行手势时频图的特征提取，从而得到体现手势特征的特征向量，进而用于手势的分类识别.本算法不仅有效解决了实际应用中有物体遮挡场景下的识别问题，还提出了系统状态转换机（System status Transition Machine，STM）和数据增强方法，实现了对手势开始和结束时间的精准控制，从而完成了手势的自动分割与实时识别.最终，在搭载Android系统的边缘计算设备上进行了系统部署，并在遮挡场景下进行测试.实验结果表明：所提出的手势识别系统能够在边缘设备上高效、准确地完成手势分类，具有较强的实用价值和应用前景.

Abstract

To address the real-time gesture recognition problem in occluded environments

this paper proposes a real-time gesture recognition algorithm based on long range radio (LoRa) signals. By utilizing the low frequency band and good penetration of LoRa signals

this algorithm calculates the signal ratio using two receiving antennas

and combines short-time Fourier transform (STFT) to obtain time-frequency maps containing hand motion features. These maps are processed by a neural network encoder called the Gesture Encoder

generating feature vectors that represent the gesture characteristics

which are then used for gesture classification and recognition. This algorithm effectively solves the recognition problem in scenarios with object occlusion

and introduces a system state transition machine (STM) and data augmentation methods to precisely control the start and end times of gestures

thus enabling automatic segmentation and real-time recognition. At last

the system is deployed on an edge computing device running Android

and tested in occlusion scenario. Experimental results show that the proposed gesture recognition system can efficiently and accurately complete gesture classification on edge devices

with strong practical value and application prospects.

关键词

Keywords

references

QIU X R , LIU J H , SONG L L , et al . A survey of gesture recognition using frequency modulated continuous wave radar [J ] . Journal of Computer and Communications , 2024 , 12 ( 6 ): 115 - 134 .

AL FARID F , HASHIM N , ABDULLAH J B , et al . Single shot detector CNN and deep dilated masks for vision-based hand gesture recognition from video sequences [J ] . IEEE Access , 2024 , 12 : 28564 - 28574 .

QI J , MA L , CUI Z C , et al . Computer vision-based hand gesture recognition for human-robot interaction: A review [J ] . Complex & Intelligent Systems , 2024 , 10 ( 1 ): 1581 - 1606 .

ZERROUKI N , HARROU F , HOUACINE A , et al . Deep learning for hand gesture recognition in virtual museum using wearable vision sensors [J ] . IEEE Sensors Journal , 2024 , 24 ( 6 ): 8857 - 8869 .

ALEMU M Y , LIN Y , SHULL P B . EchoGest: Soft ultrasonic waveguides based sensing skin for subject-independent hand gesture recognition [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2024 , 32 : 2366 - 2375 .

JIN B , MA X , HU B J , et al . Gesture-mmWAVE: Compact and accurate millimeter-wave radar-based dynamic gesture recognition for embedded devices [J ] . IEEE Transactions on Human-Machine Systems , 2024 , 54 ( 3 ): 337 - 347 .

OH H , BHATTACHARYA S , SEO S . UWB/IMU-assisted gesture recognition using learning approaches for VR/XR applications [C ] // ICC 2024 - IEEE International Conference on Communications . Piscataway : IEEE , 2024 : 4838 - 4844 .

ZHANG S C , WANG Q J , GAN M L , et al . RadSee: See your handwriting through walls using FMCW radar [C ] // Proceedings 2025 Network and Distributed System Security Symposium . Internet Society , 2025 : 112 - 123 .

ZHAO L Q , XIAO R , LIU J W , et al . One is enough: Enabling one-shot device-free gesture recognition with COTS WiFi [C ] // IEEE INFOCOM 2024 - IEEE Conference on Computer Communications . Piscataway : IEEE , 2024 : 1231 - 1240 .

GAO D M , WANG H Y , WANG S , et al . WiLo: Long-range cross-technology communication from Wi-Fi to LoRa [J ] . IEEE Transactions on Communications , 2025 , 73 ( 3 ): 1677 - 1691 .

ZHANG F S , CHANG Z X , XIONG J , et al . Exploring LoRa for sensing [J ] . GetMobile: Mobile Computing and Communications , 2021 , 25 ( 2 ): 33 - 37 .

GE Y , LI W D , FAROOQ M , et al . LoGait: LoRa sensing system of human gait recognition using dynamic time warping [J ] . IEEE Sensors Journal , 2023 , 23 ( 18 ): 21687 - 21697 .

HUANG Q Y , LUO Z Q , ZHANG J , et al . LoRadar: Enabling concurrent radar sensing and LoRa communication [J ] . IEEE Transactions on Mobile Computing , 2022 , 21 ( 6 ): 2045 - 2057 .

XIE B B , GANESAN D , XIONG J . Embracing LoRa sensing with device mobility [C ] // Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems . New York : ACM , 2022 : 349 - 361 .

XIE B B , XIONG J . Combating interference for long range LoRa sensing [C ] // Proceedings of the 18th Conference on Embedded Networked Sensor Systems . New York : ACM , 2020 : 69 - 81 .

ZHANG F S , CHANG Z X , NIU K , et al . Exploring LoRa for long-range through-wall sensing [J ] . Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies , 2020 , 4 ( 2 ): 1 - 27 .

ZHANG F S , CHANG Z X , XIONG J , et al . Unlocking the beamforming potential of LoRa for long-range multi-target respiration sensing [J ] . Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies , 2021 , 5 ( 2 ): 1 - 25 .

BERNI A , GREGG W . On the utility of chirp modulation for digital signaling [J ] . IEEE Transactions on Communications , 1973 , 21 ( 6 ): 748 - 751 .

WANG W , LIU A X , SHAHZAD M , et al . Understanding and modeling of WiFi signal based human activity recognition [C ] // Proceedings of the 21st Annual International Conference on Mobile Computing and Networking . New York : ACM , 2015 : 65 - 76 .

WANG H , ZHANG D Q , MA J Y , et al . Human respiration detection with commodity WiFi devices: Do user location and body orientation matter [C ] // Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing . New York : ACM , 2016 : 25 - 36 .

GE J F , ZHU D S , SUN L J , et al . A long-range signal-based target localization algorithm [J ] . Electronics , 2024 , 13 ( 6 ): 1069 .

韩崇 , 韩磊 , 孙力娟 , 等 . 基于时空压缩特征表示学习的毫米波雷达手势识别算法 [J ] . 电子与信息学报 , 2022 , 44 ( 4 ): 1274 - 1283 .

HAN C , HAN L , SUN L J , et al . Millimeter wave radar gesture recognition algorithm based on spatio-temporal compression feature representation learning [J ] . Journal of Electronics & Information Technology , 2022 , 44 ( 4 ): 1274 - 1283 . (in Chinese)

王勇 , 王沙沙 , 田增山 , 等 . 基于FMCW雷达的双流融合神经网络手势识别方法 [J ] . 电子学报 , 2019 , 47 ( 7 ): 1408 - 1415 .

WANG Y , WANG S S , TIAN Z S , et al . Two-stream fusion neural network approach for hand gesture recognition based on FMCW radar [J ] . Acta Electronica Sinica , 2019 , 47 ( 7 ): 1408 - 1415 . (in Chinese)

SANDLER M , HOWARD A , ZHU M L , et al . MobileNetV2: Inverted residuals and linear bottlenecks [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 4510 - 4520 .

QIAN S Y , NING C R , HU Y P . MobileNetV3 for image classification [C ] // 2021 IEEE 2nd International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE) . Piscataway : IEEE , 2021 : 490 - 497 .

SCHROFF F , KALENICHENKO D , PHILBIN J . FaceNet: A unified embedding for face recognition and clustering [C ] // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2015 : 815 - 823 .

ZHU D S , HAN C , GUO J , et al . TWLip: Exploring through-wall word-level lip reading based on coherent SISO radar [J ] . IEEE Internet of Things Journal , 2024 , 11 ( 19 ): 32310 - 32323 .

LIU H P , ZHOU A F , DONG Z H , et al . M-gesture: Person-independent real-time in-air gesture recognition using commodity millimeter wave radar [J ] . IEEE Internet of Things Journal , 2022 , 9 ( 5 ): 3397 - 3415 .

SUN Y L , FEI T , LI X B , et al . Real-time radar-based gesture detection and recognition built in an edge-computing platform [J ] . IEEE Sensors Journal , 2020 , 20 ( 18 ): 10706 - 10716 .

VAEZI Y , VAN DER BAAN M . Comparison of the STA/LTA and power spectral density methods for microseismic event detection [J ] . Geophysical Journal International , 2015 , 203 ( 3 ): 1896 - 1908 .

Alientek Technology . RK3568 Dev board [EB/OL ] . ( 2023-01-01 )[ 2024-12-22 ] . https://www.alientek.com https://www.alientek.com .

Rockchip Technology . RK3568 brief datasheet [EB/OL ] . ( 2022-01-01 )[ 2024-12-22 ] . https://www.rock-chips.com/uploads/pdf/2022.8.26/192/RK3568%20Brief%20Datasheet.pdf https://www.rock-chips.com/uploads/pdf/2022.8.26/192/RK3568%20Brief%20Datasheet.pdf .

Research Ettus . UHD (Universal Hardware Driver) [EB/OL ] . ( 2023-01-01 )[ 2024-12-22 ] . https://github.com/EttusResearch/uhd https://github.com/EttusResearch/uhd .

MORITZ-WUNDKE . Boost for android [EB/OL ] . ( 2023-01-01 )[ 2024-12-22 ] . https://github.com/moritz-wundke/Boost-for-Android https://github.com/moritz-wundke/Boost-for-Android .

LIBUSB . Libusb for android [EB/OL ] . ( 2023-01-01 )[ 2024-12-22 ] . https://github.com/libusb/libusb https://github.com/libusb/libusb .

Android , Inc . Android NDK [EB/OL ] . ( 2022-01-01 )[ 2024-12-22 ] . https://developer.android.com/ndk https://developer.android.com/ndk .

Rockchip Technology . KNN-toolkit2 [EB/OL ] . ( 2024-01-01 )[ 2024-12-22 ] . https://github.com/airockchip/rknn-toolkit2 https://github.com/airockchip/rknn-toolkit2 .

Microchip Technology . LoRa transceiver [EB/OL ] . ( 2006-01-01 )[ 2024-12-22 ] . https://microchip.ua/wireless/RF96_97_98.pdf https://microchip.ua/wireless/RF96_97_98.pdf .

Arduino . Arduino UNO rev3 [EB/OL ] . ( 2015-01-01 )[ 2024-12-22 ] . https://store.arduino.cc/usa/arduino-uno-rev3 https://store.arduino.cc/usa/arduino-uno-rev3 .

Research Ettus . USRP B210 [EB/OL ] . ( 2013-01-01 )[ 2024-12-22 ] . https://www.ettus.com/all-products/UB210-KIT https://www.ettus.com/all-products/UB210-KIT .

CHANDRAWI . LoRaRF-arduino [EB/OL ] . ( 2021-01-01 )[ 2024-12-22 ] . https://github.com/chandrawi/LoRaRF-Arduino https://github.com/chandrawi/LoRaRF-Arduino .

陈君毅 , 蒋德琛 , 王智铭 , 等 . 一种基于双维度滤波和自适应定长化的FMCW雷达手势识别算法研究 [J ] . 电子学报 , 2023 , 51 ( 8 ): 2179 - 2187 .

CHEN J Y , JIANG D C , WANG Z M , et al . Research on a gesture recognition algorithm for FMCW radar based on bidimensional filtering and adaptive fixed length [J ] . Acta Electronica Sinica , 2023 , 51 ( 8 ): 2179 - 2187 . (in Chinese)

韩崇 , 樊卫北 , 郭澳 . 基于特征融合的毫米波雷达行为识别算法 [J ] . 计算机科学 , 2024 , 51 ( 12 ): 181 - 189 .

HAN C , FAN W B , GUO A . Millimeter wave radar human activity recognition algorithm based on feature fusion [J ] . Computer Science , 2024 , 51 ( 12 ): 181 - 189 . (in Chinese)

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C ] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2016 : 770 - 778 .

GAN L Y , LIU Y , LI Y Z , et al . Gesture recognition system using 24 GHz FMCW radar sensor realized on real-time edge computing platform [J ] . IEEE Sensors Journal , 2022 , 22 ( 9 ): 8904 - 8914 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Neural Network Approach to Computing the Eigenvectors Corresponding to the Largest and Smallest Eigenvalues of a Positive Matrix

Continual Learning Methods and Applications in Computer Vision

ECG Motion Artifact Filtering Based on Two-Stage Adaptive Threshold Rules

End-to-End Full Complex-Valued Domain SAR Target Classification Neural Network

Related Author

罗发龙

李衍达

FANG Yan

Wei Yun-chao

Cong Run-min

Zuo Wang-meng

Zhao Yao

LÜ Jian-hang

Related Institution

Department of Automation,Tsinghua University

School of Computer Science and Technology, Beijing Jiaotong University

School of Control Science and Engineering, Shandong University

School of Computer Science and Technology, Harbin Institute of Technology

College of Electrical Engineering and Automation， Fuzhou University

⁰