An RGB-Event Object Tracking Method Based on Geometry Prior Guidance

ZHAO Shaochuan; LIU Zhengwei; XU Tianyang; SHAO Zhiwen; ZHOU Yong; WU Xiaojun

doi:10.12263/DZXB.20251160

您当前的位置：

首页 >

文章列表页 >

An RGB-Event Object Tracking Method Based on Geometry Prior Guidance

PAPERS | 更新时间：2026-07-30

- An RGB-Event Object Tracking Method Based on Geometry Prior Guidance
- ACTA ELECTRONICA SINICA Vol. 54, Issue 4, Pages: 1719-1731(2026)
- 作者机构：
  
  1.中国矿业大学计算机科学与技术学院/人工智能学院，江苏徐州 221116
  2.矿山数字化教育部工程研究中心，江苏徐州 221116
  3.地下空间智能感知与应急物联江苏省产业技术工程化中心，江苏徐州 221116
  4.江南大学人工智能与计算机学院，江苏无锡 214122
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62272461;62020106012;62332008;62336004;62472424);Fundamental Research Funds for the Central Universities(2025QN1162;JUSRP202504007)
- DOI：10.12263/DZXB.20251160
  CLC： TN911.73;
- Received：26 January 2026，
  
  Accepted：19 March 2026，
  
  Online First：23 April 2026，
  
  Published：25 April 2026
- 稿件说明：
移动端阅览
赵少川, 刘峥玮, 徐天阳, 等. 基于几何先验引导的可见光-事件目标跟踪方法[J]. 电子学报, 2026, 54(04): 1719-1731.

ZHAO Shaochuan, LIU Zhengwei, XU Tianyang, et al. An RGB-Event Object Tracking Method Based on Geometry Prior Guidance[J]. Acta Electronica Sinica, 2026, 54(04): 1719-1731.
赵少川, 刘峥玮, 徐天阳, 等. 基于几何先验引导的可见光-事件目标跟踪方法[J]. 电子学报, 2026, 54(04): 1719-1731. DOI：10.12263/DZXB.20251160

ZHAO Shaochuan, LIU Zhengwei, XU Tianyang, et al. An RGB-Event Object Tracking Method Based on Geometry Prior Guidance[J]. Acta Electronica Sinica, 2026, 54(04): 1719-1731. DOI：10.12263/DZXB.20251160

摘要

可见光相机在运动模糊、极端光照等挑战性场景下易出现成像质量退化，导致现有基于可见光模态的目标跟踪方法性能受限。事件相机凭借高时间分辨率、高动态范围和低延迟等优势，可有效弥补可见光传感器的不足，因此可见光-事件目标跟踪逐渐成为研究热点。然而，现有方法高度依赖事件相机的成像质量，当目标处于静止或低速运动状态时，事件信号因缺乏光强变化而难以提供有效的鉴别性信息，制约了跟踪器在复杂场景下的鲁棒性。针对上述问题，本文提出一种基于几何先验引导的可见光-事件目标跟踪方法。考虑到事件信号与角点/边缘信号均对目标边缘区域敏感，且可见光图像在事件信号缺失时可提供稳定的空间几何结构信息，本文利用角点与边缘作为几何先验引导信号，以增强事件模态的表达能力。为实现有效融合，本文设计了两种几何先验引导策略：（1）像素级引导策略，将提取的角点/边缘灰度图与事件图像在通道维度拼接，直接作为跟踪模型输入；（2）特征级引导策略，分别提取事件图像与角点/边缘图像的深度特征，通过信号质量评估模块自适应学习融合权重，实现特征的动态加权融合。两种策略均具备良好的兼容性，可便捷嵌入现有可见光-事件跟踪架构，并以较低计算开销提升模型在事件模态成像质量不佳时的鲁棒性。本文将所提方法集成至TENet（Targetness Entanglement Network）、ViPT（Visual Prompt multi-modal Tracking）与SDSTrack（Self-Distillation Symmetric Tracking）3种先进可见光-事件跟踪模型中，在VisEvent、COESOT和FE240hz三个公开数据集上开展实验验证。实验结果表明，几何先验引导策略显著提升了跟踪精度，尤其在目标静止、形变、部分遮挡等挑战性场景下改善明显。以TENet为例，在FE240hz数据集上的成功率SUC（SUCcess rate）和精确率PRE（PREcision rate）分别提升了1.6%和2.3%；在涵盖多种跟踪场景的COESOT数据集上的两种指标分别提升了0.8%和1.2%；在VisEvent数据集上也分别获得了0.4%和0.5%的增益。综上，本文提出的几何先验引导方法通过引入角点/边缘信息作为事件信号的有效补充，缓解了事件相机在低动态场景下的信息缺失问题，为可见光-事件目标跟踪提供了一种高效且鲁棒的解决方案。

Abstract

RGB cameras often suffer from image quality degradation in challenging scenarios such as motion blur and extreme lighting conditions

which limits the performance of existing RGB-based tracker. Event cameras

with their advantages of high temporal resolution

high dynamic range

and low latency

can effectively compensate for the shortcomings of RGB sensors

making RGB-Event object tracking a gradually emerging research focus. However

existing methods heavily rely on the imaging quality of event cameras. When the object is stationary or moving at low speeds

event signals fail to provide effective discriminative information due to the lack of intensity changes

hereby constraining the robustness of trackers in complex scenarios. To address the above issues

this paper proposes an RGB-Event object tracking method guided by geometric priors. Considering that both event signals and corner/edge signals are sensitive to object edge regions

and that RGB images can provide stable spatial geometric structure information in the absence of event signals

this paper employs corners and edges as geometric prior guidance signals to enhance the representational capability of the event modality. To achieve effective fusion

two geometric prior guidance strategies are designed: 1) a pixel-level guidance strategy

which concatenates the extracted corner/edge grayscale maps with event images along the channel dimension as direct input to the tracking model; and 2) a feature-level guidance strategy

which separately extracts high-level features from event images and corner/edge maps

and adaptively learns fusion weights through a signal quality assessment module to achieve dynamic weighted fusion of features. Both strategies offer good compatibility

can be conveniently integrated into existing RGB-Event trackers

enhancing tracking robustness under poor event modality imaging quality with low computational overhead. The proposed method is integrated into three advanced RGB-Event trackers

namely TENet

ViPT

and SDSTrack

and experimental validation is conducted on three public datasets: VisEvent

COESOT

and FE240hz. The results demonstrate that the geometric prior guidance strategy significantly improves tracking accuracy

particularly in challenging scenarios such as object stationary

deformation

and partial occlusion. Taking TENet as an example

the success rate(SUC) and precision rate(PRE) on the FE240hz dataset are improved by 1.6% and 2.3%

respectively; on the COESOT dataset

which covers various tracking scenarios

the two metrics are improved by 0.8% and 1.2%

respectively; and on the VisEvent dataset

gains of 0.4% and 0.5% are achieved

respectively. In summary

the propos

ed geometric prior guidance method effectively alleviates the information deficiency problem of event cameras in low-dynamic scenarios by introducing corner/edge information as a complement to event signals

providing an efficient and robust solution for RGB-Event object tracking. Our codes are at

https://github.com/LLLzw444/Enhancement-strategy-of-TENet

关键词

Keywords

references

谷美颖 , 李航 , 张家伟 , 等 . 基于视觉的无人机定位与导航方法研究综述 [J ] . 电子学报 , 2025 , 53 ( 3 ): 651 - 685 .

Gu Meiying , Li Hang , Zhang Jiawei , et al . A review of vision-based UAV localization and navigation methods [J ] . Acta Electronica Sinica , 2025 , 53 ( 3 ): 651 - 685 . (in Chinese)

钟钰彬 , 杨鹏 , 窦磊 . 基于纵横比自适应的相关滤波跟踪算法 [J ] . 电子学报 , 2024 , 52 ( 6 ): 2112 - 2122 .

Zhong Yubin , Yang Peng , Dou Lei . Correlation filtering tracking algorithm based on adaptive aspect-ratio [J ] . Acta Electronica Sinica , 2024 , 52 ( 6 ): 2112 - 2122 . (in Chinese)

李玺 , 查宇飞 , 张天柱 , 等 . 深度学习的目标跟踪算法综述 [J ] . 中国图象图形学报 , 2019 , 24 ( 12 ): 2057 - 2080 . DOI: 10.11834/jig.190372 http://dx.doi.org/10.11834/jig.190372

Li Xi , Zha Yufei , Zhang Tianzhu , et al . Survey of visual object tracking algorithms based on deep learning [J ] . Journal of Image and Graphics , 2019 , 24 ( 12 ): 2057 - 2080 . (in Chinese) . DOI: 10.11834/jig.190372 http://dx.doi.org/10.11834/jig.190372

林慧兰 , 赵春蕾 , 郝志成 , 等 . 复杂场景单目标跟踪 [J ] . 光学精密工程 , 2024 , 32 ( 23 ): 3490 - 3503 . DOI: 10.37188/ope.20243223.3490 http://dx.doi.org/10.37188/ope.20243223.3490

Lin Huilan , Zhao Chunlei , Hao Zhicheng , et al . Single target tracking in complex scenarios [J ] . Optics and Precision Engineering , 2024 , 32 ( 23 ): 3490 - 3503 . (in Chinese) . DOI: 10.37188/ope.20243223.3490 http://dx.doi.org/10.37188/ope.20243223.3490

陈刚 , 张文标 , 王洪雁 , 等 . 三维激光雷达点云级联匹配多目标跟踪方法 [J ] . 光电工程 , 2025 , 52 ( 6 ): 250050 .

Chen Gang , Zhang Wenbiao , Wang Hongyan , et al . The cascade matching method for multi-object tracking of 3D LiDAR point clouds [J ] . Opto-Electronic Engineering , 2025 , 52 ( 6 ): 250050 . (in Chinese)

刘超军 , 段喜萍 , 谢宝文 . 应用GhostNet卷积特征的ECO目标跟踪算法改进 [J ] . 激光技术 , 2022 , 46 ( 2 ): 239 - 247 . DOI: 10.7510/jgjs.issn.1001-3806.2022.02.015 http://dx.doi.org/10.7510/jgjs.issn.1001-3806.2022.02.015

Liu Chaojun , Duan Xiping , Xie Baowen . Improvement of ECO target tracking algorithm based on GhostNet convolution feature [J ] . Laser Technology , 2022 , 46 ( 2 ): 239 - 247 . (in Chinese) . DOI: 10.7510/jgjs.issn.1001-3806.2022.02.015 http://dx.doi.org/10.7510/jgjs.issn.1001-3806.2022.02.015

Bertinetto L , Valmadre J , Henriques J F , et al . Fully-convolutional Siamese networks for object tracking [C ] // Proceedings of the European Conference on Computer Vision . Heidelberg : Springer , 2016 : 850 - 865 . DOI: 10.1007/978-3-319-48881-3_56 http://dx.doi.org/10.1007/978-3-319-48881-3_56

Li Bo , Yan Junjie , Wu Wei , et al . High performance visual tracking with Siamese region proposal network [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 8971 - 8980 . DOI: 10.1109/cvpr.2018.00935 http://dx.doi.org/10.1109/cvpr.2018.00935

Chen Xin , Yan Bin , Zhu Jiawen , et al . Transformer tracking [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2021 : 8122 - 8131 . DOI: 10.1109/cvpr46437.2021.00803 http://dx.doi.org/10.1109/cvpr46437.2021.00803

Ye Botao , Chang Hong , Ma Bingpeng , et al . Joint feature learning and relation modeling for tracking: A one-stream framework [C ] // Proceedings of the 17th European Conference on Computer Vision . Heidelberg : Springer , 2022 : 341 - 357 . DOI: 10.1007/978-3-031-20047-2_20 http://dx.doi.org/10.1007/978-3-031-20047-2_20

Wang Xiao , Li Jianing , Zhu Lin , et al . VisEvent: Reliable object tracking via collaboration of frame and event flows [J ] . IEEE Transactions on Cybernetics , 2024 , 54 ( 3 ): 1997 - 2010 . DOI: 10.1109/tcyb.2023.3318601 http://dx.doi.org/10.1109/tcyb.2023.3318601

Gehrig D , Scaramuzza D . Low-latency automotive vision with event cameras [J ] . Nature , 2024 , 629 ( 8014 ): 1034 - 1040 . DOI: 10.1038/s41586-024-07409-w http://dx.doi.org/10.1038/s41586-024-07409-w

Shao Pengcheng , Xu Tianyang , Tang Zhangyong , et al . TENet: Targetness entanglement incorporating with multi-scale pooling and mutually-guided fusion for RGB-E object tracking [J ] . Neural Networks , 2025 , 183 : 106948 . DOI: 10.1016/j.neunet.2024.106948 http://dx.doi.org/10.1016/j.neunet.2024.106948

Zhu Jiawen , Lai Simiao , Chen Xin , et al . Visual prompt multi-modal tracking [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2023 : 9516 - 9526 . DOI: 10.1109/cvpr52729.2023.00918 http://dx.doi.org/10.1109/cvpr52729.2023.00918

Hou Xiaojun , Xing Jiazheng , Qian Yijie , et al . SDSTrack: Self-distillation symmetric adapter learning for multi-modal visual object tracking [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2024 : 26541 - 26551 . DOI: 10.1109/cvpr52733.2024.02507 http://dx.doi.org/10.1109/cvpr52733.2024.02507

Wu Zongwei , Zheng Jilai , Ren Xiangxuan , et al . Single-model and any-modality for video object tracking [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2024 : 19156 - 19166 . DOI: 10.1109/cvpr52733.2024.01812 http://dx.doi.org/10.1109/cvpr52733.2024.01812

Tang Chuanming , Wang Xiao , Huang Ju , et al . Revisiting color-event based tracking: A unified network, dataset, and metric [J ] . Pattern Recognition , 2026 , 172 : 112718 . DOI: 10.1016/j.patcog.2025.112718 http://dx.doi.org/10.1016/j.patcog.2025.112718

Zhang Jiqing , Yang Xin , Fu Yingkai , et al . Object tracking by jointly exploiting frame and event domain [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE , 2021 : 13023 - 13032 . DOI: 10.1109/iccv48922.2021.01280 http://dx.doi.org/10.1109/iccv48922.2021.01280

Li Bo , Wu Wei , Wang Qiang , et al . SiamRPN++: Evolution of Siamese visual tracking with very deep networks [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 4282 - 4291 . DOI: 10.1109/cvpr.2019.00441 http://dx.doi.org/10.1109/cvpr.2019.00441

吴非 , 张建林 . 结合全局光流的孪生区域提名网络目标跟踪算法 [J ] . 半导体光电 , 2023 , 44 ( 3 ): 422 - 428 .

Wu Fei , Zhang Jianlin . Siamese region proposal network object tracking algorithm with global optical flow [J ] . Semiconductor Optoelectronics , 2023 , 44 ( 3 ): 422 - 428 . (in Chinese)

Wang Qiang , Zhang Li , Bertinetto L , et al . Fast online object tracking and segmentation: A unifying approach [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2019 : 1328 - 1338 . DOI: 10.1109/cvpr.2019.00142 http://dx.doi.org/10.1109/cvpr.2019.00142

谢青松 , 刘晓庆 , 安志勇 , 等 . 基于前景优化的视觉目标跟踪算法 [J ] . 电子学报 , 2022 , 50 ( 7 ): 1558 - 1566 .

Xie Qingsong , Liu Xiaoqing , An Zhiyong , et al . Visual object tracking algorithm based on foreground optimization [J ] . Acta Electronica Sinica , 2022 , 50 ( 7 ): 1558 - 1566 . (in Chinese)

Chen Zedu , Zhong Bineng , Li Guorong , et al . SiamBAN: Target-aware tracking with Siamese box adaptive network [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023 , 45 ( 4 ): 5158 - 5173 .

Yan Bin , Peng Houwen , Fu Jianlong , et al . Learning spatio-temporal transformer for visual tracking [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE , 2021 : 10428 - 10437 . DOI: 10.1109/iccv48922.2021.01028 http://dx.doi.org/10.1109/iccv48922.2021.01028

Tan Yuedong , Wu Zongwei , Fu Yuqian , et al . XTrack: Multimodal training boosts RGB-X video object trackers [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE , 2025 : 5734 - 5744 . DOI: 10.1109/iccv51701.2025.00543 http://dx.doi.org/10.1109/iccv51701.2025.00543

Gallego G , Delbrück T , Orchard G , et al . Event-based vision: A survey [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 1 ): 154 - 180 . DOI: 10.1109/tpami.2020.3008413 http://dx.doi.org/10.1109/tpami.2020.3008413

Zhang Jiqing , Wang Yuanchen , Liu Wenxi , et al . Frame-event alignment and fusion network for high frame rate tracking [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2023 : 9781 - 9790 . DOI: 10.1109/cvpr52729.2023.00943 http://dx.doi.org/10.1109/cvpr52729.2023.00943

Harris C , Stephens M . A combined corner and edge detector [C ] // Proceedings of the Alvey Vision Conference . Manchester : Alvey Vision Club , 1988 : 147 - 152 . DOI: 10.5244/c.2.23 http://dx.doi.org/10.5244/c.2.23

Lowe D G . Distinctive image features from scale-invariant keypoints [J ] . International Journal of Computer Vision , 2004 , 60 ( 2 ): 91 - 110 . DOI: 10.1023/b:visi.0000029664.99615.94 http://dx.doi.org/10.1023/b:visi.0000029664.99615.94

Alcantarilla P F , Bartoli A , Davison A J . KAZE features [C ] // Proceedings of the 12th European Conference on Computer Vision . Heidelberg : Springer , 2012 : 214 - 227 . DOI: 10.1007/978-3-642-33783-3_16 http://dx.doi.org/10.1007/978-3-642-33783-3_16

Rublee E , Rabaud V , Konolige K , et al . ORB: An efficient alternative to SIFT or SURF [C ] // Proceedings of the 2011 International Conference on Computer Vision . Piscataway : IEEE , 2011 : 2564 - 2571 . DOI: 10.1109/iccv.2011.6126544 http://dx.doi.org/10.1109/iccv.2011.6126544

Rosten E , Drummond T . Machine learning for high-speed corner detection [C ] // Proceedings of the 9th European Conference on Computer Vision . Heidelberg : Springer , 2006 : 430 - 443 . DOI: 10.1007/11744023_34 http://dx.doi.org/10.1007/11744023_34

Calonder M , Lepetit V , Strecha C , et al . BRIEF: Binary robust independent elementary features [C ] // Proceedings of the 11th European Conference on Computer Vision . Heidelberg : Springer , 2010 : 778 - 792 . DOI: 10.1007/978-3-642-15561-1_56 http://dx.doi.org/10.1007/978-3-642-15561-1_56

Kanopoulos N , Vasanthavada N , Baker R L . Design of an image edge detection filter using the Sobel operator [J ] . IEEE Journal of Solid-State Circuits , 1988 , 23 ( 2 ): 358 - 367 . DOI: 10.1109/4.996 http://dx.doi.org/10.1109/4.996

Marr D , Hildreth E . Theory of edge detection [J ] . Proceedings of the Royal Society of London. Series B. Biological Sciences , 1980 , 207 ( 1167 ): 187 - 217 . DOI: 10.1098/rspb.1980.0020 http://dx.doi.org/10.1098/rspb.1980.0020

Canny J . A computational approach to edge detection [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 1986 , PAMI-8( 6 ): 679 - 698 . DOI: 10.1109/tpami.1986.4767851 http://dx.doi.org/10.1109/tpami.1986.4767851

贾迪 , 孟祥福 , 孟琭 , 等 . RGB空间下结合高斯曼哈顿距离图的彩色图像边缘检测 [J ] . 电子学报 , 2014 , 42 ( 2 ): 257 - 263 .

Jia Di , Meng Xiangfu , Meng Lu , et al . Color image edge detection combining with Gauss Manhattan distance map in RGB space [J ] . Acta Electronica Sinica , 2014 , 42 ( 2 ): 257 - 263 . (in Chinese)

Yang Jinyu , Li Zhe , Zheng Feng , et al . Prompting for multi-modal tracking [C ] // Proceedings of the 30th ACM International Conference on Multimedia . New York : ACM , 2022 : 3492 - 3500 . DOI: 10.1145/3503161.3547851 http://dx.doi.org/10.1145/3503161.3547851

Danelljan M , Van Gool L , Timofte R . Probabilistic regression for visual tracking [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2020 : 7181 - 7190 . DOI: 10.1109/cvpr42600.2020.00721 http://dx.doi.org/10.1109/cvpr42600.2020.00721

Wang Xiao , Wang Shiao , Tang Chuanming , et al . Event stream-based visual object tracking: A high-resolution benchmark dataset and a novel baseline [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2024 : 19248 - 19257 . DOI: 10.1109/cvpr52733.2024.01821 http://dx.doi.org/10.1109/cvpr52733.2024.01821

Zhang Tianlu , Debattista K , Zhang Qiang , et al . Revisiting motion information for RGB-event tracking with MOT philosophy [C ] // Proceedings of the 38th International Conference on Neural Information Processing System . New York : Curran Associates Inc. , 2024 : 2836 . DOI: 10.52202/079017-2836 http://dx.doi.org/10.52202/079017-2836

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Dual-Domain Feature Interaction and Local Correlation Upsampling Network for Single Image Dehazing

Research Progress of UAV Autonomous Landing Scene Perception Methods Based on Vision and Deep Learning

Visual Object Tracking Algorithm Based on Adaptive Feature Selection

Low-Light Small Target Detection Method Combining Feature Fusion Enhancement and Detail Features

EnvAct: An IoT Automation Execution Framework Based on Large Language Models and Structured Environment Representation

Related Author

LIU Ming-jie

LYU Meng-lin

LIU Ping

CHEN Jun-sheng

PIAO Chang-hao

KANG Zong-xu

WANG Zhong-tian

WU Yi-quan

Related Institution

School of Automation, Chongqing University of Posts and Telecommunications

Aerospace Jinmei (Chongqing) Communication Co., Ltd.

College of Electronic and Information Engineering, Nanjing University of Aeronautics and Astronautics

School of Electronic Information and Engineering, Changchun University of Science and Technology

Jilin Province Technology Research Center of Photoelectric Detection and Intelligent Information Processing

⁰