电子学报 ›› 2022, Vol. 50 ›› Issue (7): 1558-1566.DOI: 10.12263/DZXB.20210641
谢青松1, 刘晓庆2, 安志勇1, 李博1
收稿日期:
2021-05-19
修回日期:
2021-10-07
出版日期:
2022-07-25
作者简介:
基金资助:
XIE Qing-song1, LIU Xiao-qing2, AN Zhi-yong1, LI Bo1
Received:
2021-05-19
Revised:
2021-10-07
Online:
2022-07-25
Published:
2022-07-30
Supported by:
摘要:
将目标分割技术引入跟踪领域是当前的研究热点.目前,基于分割的跟踪算法往往根据分割结果计算最小外接矩形,以此作为跟踪框,但复杂的目标运动使得跟踪框内包含较多背景,从而导致精度下降.针对该问题,本文提出了一种基于前景优化的视觉目标跟踪算法,将跟踪框的尺度和角度优化统一于前景优化框架中.首先评估跟踪框内的前景比例,若小于设定阈值,则对跟踪框分别进行尺度和角度优化;在尺度优化模块中,结合回归框计算跟踪框的条件概率,根据条件概率的结果分情形进行尺度优化;角度优化模块中,针对跟踪框设定多个偏移角度,利用前景IoU(Intersection over Union)极大策略选择最优跟踪框角度.结果证明,将本文方法应用于SiamMask算法,精度在VOT2016,VOT2018和VOT2019数据集分别提升约3.2%,3.7%和3.6%,而EAO分别提升约1.8%,1.9%和1.6%.另外,本文的方法针对基于分割的跟踪算法具有一定的普适性.
中图分类号:
谢青松, 刘晓庆, 安志勇, 李博. 基于前景优化的视觉目标跟踪算法[J]. 电子学报, 2022, 50(7): 1558-1566.
XIE Qing-song, LIU Xiao-qing, AN Zhi-yong, LI Bo. Visual Object Tracking Algorithm Based on Foreground Optimization[J]. Acta Electronica Sinica, 2022, 50(7): 1558-1566.
图1 基于前景优化的视觉目标跟踪算法整体流程框架.图中的R为回归分支输出的回归框,R′为旋转后的回归框,F为目标分割掩码,M为掩码的最小外接矩形,T为尺度优化后输出的跟踪框,T?i为角度优化后输出的最终跟踪框,AS(Adaptive Strategy)为自适应策略,MS(Mean Strategy)为均值策略,AOS(Angle Offset Strategy)为角度偏移策略.
VOT2016 | VOT2018 | VOT2019 | 平均速度(fps) | |||||||
---|---|---|---|---|---|---|---|---|---|---|
A↑ | R↓ | EAO↑ | A↑ | R↓ | EAO↑ | A↑ | R↓ | EAO↑ | ||
SiamMask | 0.620 | 0.214 | 0.436 | 0.597 | 0.248 | 0.406 | 0.596 | 0.467 | 0.283 | 88 |
SiamMask_E | 0.645 | 0.210 | 0.452 | 0.627 | 0.248 | 0.427 | 0.625 | 0.482 | 0.298 | — |
SiamMask+SO(MS) | 0.644 | 0.228 | 0.437 | 0.623 | 0.267 | 0.410 | 0.622 | 0.487 | 0.294 | 88 |
SiamMask+SO(AS) | 0.654 | 0.233 | 0.439 | 0.634 | 0.276 | 0.417 | 0.633 | 0.502 | 0.298 | 88 |
SiamMask+SO(MS)+AO | 0.642 | 0.219 | 0.443 | 0.622 | 0.258 | 0.415 | 0.619 | 0.482 | 0.294 | 75 |
SiamMask+SO(AS)+AO | 0.652 | 0.224 | 0.454 | 0.634 | 0.267 | 0.425 | 0.632 | 0.497 | 0.299 | 75 |
表1 三个数据集下消融实验结果(粗体为最佳性能)
VOT2016 | VOT2018 | VOT2019 | 平均速度(fps) | |||||||
---|---|---|---|---|---|---|---|---|---|---|
A↑ | R↓ | EAO↑ | A↑ | R↓ | EAO↑ | A↑ | R↓ | EAO↑ | ||
SiamMask | 0.620 | 0.214 | 0.436 | 0.597 | 0.248 | 0.406 | 0.596 | 0.467 | 0.283 | 88 |
SiamMask_E | 0.645 | 0.210 | 0.452 | 0.627 | 0.248 | 0.427 | 0.625 | 0.482 | 0.298 | — |
SiamMask+SO(MS) | 0.644 | 0.228 | 0.437 | 0.623 | 0.267 | 0.410 | 0.622 | 0.487 | 0.294 | 88 |
SiamMask+SO(AS) | 0.654 | 0.233 | 0.439 | 0.634 | 0.276 | 0.417 | 0.633 | 0.502 | 0.298 | 88 |
SiamMask+SO(MS)+AO | 0.642 | 0.219 | 0.443 | 0.622 | 0.258 | 0.415 | 0.619 | 0.482 | 0.294 | 75 |
SiamMask+SO(AS)+AO | 0.652 | 0.224 | 0.454 | 0.634 | 0.267 | 0.425 | 0.632 | 0.497 | 0.299 | 75 |
1 | MATHURG, SOMWANSHID, BUNDELEM. Intelligent video surveillance based on object tracking[C]//Proceedings of the 2018 3rd International Conference and Workshops on Recent Advances and Innovations in Engineering. Jaipur, India: IEEE Press, 2018: 1-6. |
2 | GAJJARV, KHANDHEDIYAY, GURNANIA. Human detection and tracking for video surveillance: A cognitive science approach[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops. Venice, Italy: IEEE Press, 2017: 2805-2809. |
3 | CHENB, SAHDEVR, TSOTSOSJ, et al. Person following robot using selected online ada-boosting with stereo camera[C]//Proceedings of the 2017 14th Conference on Computer and Robot Vision. Edmonton, Canada: IEEE Press, 2017: 48-55. |
4 | 张爱梅. 目标跟踪瞄准系统的虚拟现实研究[D].西安: 西安电子科技大学, 2004. |
ZHANGAi-mei. Virtual Reality Research of Target Tracking and Aiming System[D]. Xi'an: Xidian University, 2004. (in Chinese) | |
5 | AGARWALN, CHIANGC, SHARMAA. A study on computer vision techniques for self-driving cars[C]//Proceedings of the International Conference on Frontier Computing. Singapore: Springer, 2019: 629-634. |
6 | 孟琭, 杨旭. 目标跟踪算法综述[J]. 自动化学报, 2019, 45(7): 1244-1260. |
MENGLu, YANGXu. A survey of object tracking algorithms[J]. Acta Automatica Sinica, 2019, 45(7): 1244-1260. (in Chinese) | |
7 | WUY, LIM J, YANGM H. Online object tracking: A benchmark[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Portland, USA: IEEE Press, 2013: 2411- 2418. |
8 | 李玺, 查宇飞, 张天柱, 等. 深度学习的目标跟踪算法综述[J]. 中国图象图形学报, 2019, 24(12): 2057-2080. |
LIXi, ZHAYu-fei, ZHANGTian-zhu, et al. Survey of visual object tracking algorithms based on deep learning[J]. Journal of Image and Graphics, 2019, 24(12): 2057-2080. (in Chinese) | |
9 | SUND Q, ROTHS, BLACKM J. Secrets of optical flow estimation and their principles[C]//Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, CA: IEEE, 2010: 2432-2439. |
10 | XUL, JIAJ Y, MATSUSHITAY. Motion detail preserving optical flow estimation[C]//Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE, 2010: 1293-1300. |
11 | NUMMIAROK, KOLLER-MEIERE, VAN GOOLL. An adaptive color-based particle filter[J]. Image and Vision Computing, 2003, 21(1): 99-110. |
12 | DUK, JUY F, JINY L, et al. Object tracking based on improved MeanShift and SIFT[C]//Proceedings of the 2nd International Conference on Consumer Electronics, Communications and Networks. Yichang, China: IEEE, 2012: 2716-2719. |
13 | EXNERD, BRUNSE, KURZD, et al. Fast and robust CAMShift tracking[C]//Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE, 2010: 9-16. |
14 | BOLMED, BEVERIDGEJ, DRAPERB, LUI Y. Visual object tracking using adaptive correlation filters[C]//Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE, 2010: 2544-2550. |
15 | DANELLJANM, et al. Discriminative scale space tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(8): 1561-1575. |
16 | 丁新尧, 张鑫. 基于显著性特征的选择性目标跟踪算法[J]. 电子学报, 2020, 48(1): 118-123. |
DINGXin-yao, ZHANGXin. Visual tracking with salient features and selective mechanism[J]. Acta Electronica Sinica, 2020, 48(1): 118-123. (in Chinese) | |
17 | 王治丹, 蒋建国, 齐美彬, 等. 基于最大池图匹配的形变目标跟踪方法[J]. 电子学报, 2017, 45(3): 704-711. |
WANGZhi-dan, JIANGJian-guo, QIMei-bin, et al. Deformable object tracking based on max-pooling graph matching[J]. Acta Electronica Sinica, 2017, 45(3): 704-711. (in Chinese) | |
18 | WANGN Y, YEUNGD Y. Learning a deep compact image representation for visual tracking[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada: Curran Associates Inc, 2013: 809-817. |
19 | ROUTL, MISHRAD, et al. Rotation adaptive visual object tracking with motion consistency[C]//Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision(WACV). Lake Tahoe, USA: IEEE, 2018: 1047-1055. |
20 | HUAY, ALAHARIK, SCHMIDC. Online object tracking with proposal selection[C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 3092-3100. |
21 | WANGQ, ZHANGL, BERTINETTOL, et al. Fast online object tracking and segmentation: A unifying approach[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE Press, 2019: 1328-1338. |
22 | PINHEIROP O, COLLOBERTR, DOLLARP. Learning to segment object candidates[C]//Proceedings of the Neural Information Processing Systems. Montreal, Canada:ACM, 2015: 1990-1998. |
23 | PINHEIROP O, LINT, COLLOBERTR, et al. Learning to refine object segments[C]//Proceedings of the European Conference on Computer Vision. Amsterdam: Springer, 2016: 75-91. |
24 | KRISTANM, LEONARDISA, MATASJ, et al. The visual object tracking vot2016 challenge results[C]//Proceedings of the European Conference on Computer Vision. Amsterdam: Springer, 2016: 777-823. |
25 | KRISTANM, LEONARDISA, MATASJ, et al. The sixth visual object tracking vot2018 challenge results[C]//Proceedings of the European Conference on Computer Vision. Munich, Germany: Springer,2018: 3-53. |
26 | KRISTANM, LEONARDISA, MATASJ, et al. The seventh visual object tracking vot2019 challenge results[C]//Proceedings of the IEEE International Conference on Computer Vision Workshop. Seoul, Korea: IEEE, 2019: 2206-2241. |
27 | BERTINETTOL, VALMADREJ, HENRIQUESJ F, et al. Fully-convolutional Siamese networks for object tracking[C]//Proceedings of the European Conference on Computer Vision. Amsterdam: Springer,2016: 850-865. |
28 | LIB, YANJ, WUW, et al. High performance visual tracking with Siamese region proposal network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake, USA: IEEE Press, 2018: 8971-8980. |
29 | LIB, WUW, WANGQ, et al. SiamRPN++: Evolution of Siamese visual tracking with very deep networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE Press, 2018: 4277-4286. |
30 | GUOD, WANGJ, CUIY, et al. SiamCAR: Siamese fully convolutional classification and regression for visual tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 6268-6276. |
31 | CHENZ, ZHONGB, LIG, et al. Siamese box adaptive network for visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 6667-6676. |
32 | HEA, LUOC, TIANX, et al. A twofold Siamese network for real-time object tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018:4834-4843. |
33 | DUY, LIUP, ZHAOW, et al. Correlation-guided attention for corner detection based visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 6835-6844. |
34 | RENS, HEK, GIRSHICKR, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137- 1149. |
35 | CHENB, TSOTSOS JOHNK. Fast visual object tracking using ellipse fitting for rotated bounding boxes[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop(ICCVW). Seoul: IEEE, 2019: 2281-2289. |
36 | ZHUZ, WANGQ, LIB, et al. Distractor-aware Siamese networks for visual object tracking[C]//Proceedings of the IEEE European Conference on Computer Vision. Munich, Germany: Springer, 2018:103-119. |
37 | WANGG, LUOC, XIONGZ, et al. SPM-tracker: Series-parallel matching for real-time visual object tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019: 3638-3647. |
38 | VOIGTLAENDERP, LUITENETJ, et al. Siam R-CNN: Visual tracking by re-detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 6577-6587. |
39 | ZHANGZ, PENGH. Deeper and wider Siamese networks for real-time visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019: 4586-4595. |
40 | LIX, MAC, WUB, et al. Target-aware deep tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019:1369-1378. |
41 | DAIK, WANGD, LUH, et al. Visual tracking via adaptive spatially- regularized correlation filters[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019:4665-4674. |
42 | PARKE, BERGA C. Meta-tracker: Fast and robust online adaptation for visual object trackers[C]//Proceedings of the IEEE European Conference on Computer Vision. Munich, Germany: Springer, Cham, 2018:587-604. |
43 | ZHANGL, GONZALEZ-GARCIAA, WEIJERJ, et al. Learning the model update for Siamese trackers[C]//Proceedings of the IEEE International Conference on Computer Vision. Seoul, Korea: IEEE, 2019:4010-4019. |
44 | YANGT, XUP, HUR, et al. ROAM: Recurrently optimizing tracking model[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 6717-6726. |
[1] | 付利华, 赵宇, 姜涵煦, 赵茹, 吴会贤, 闫绍兴. 基于前景感知视觉注意的半监督视频目标分割[J]. 电子学报, 2022, 50(1): 195-206. |
[2] | 侯利明, 连峰, 谭顺成, 徐从安. 闪烁噪声统计特性未知情况下的鲁棒广义标签多伯努利滤波器[J]. 电子学报, 2021, 49(7): 1346-1353. |
[3] | 陈丹, 姚伯羽. 运动模型引导的自适应核相关目标跟踪方法[J]. 电子学报, 2021, 49(3): 550-558. |
[4] | 罗大鹏, 杜国庆, 曾志鹏, 魏龙生, 高常鑫, 陈应, 肖菲, 罗琛. 基于少量样本学习的多目标检测跟踪方法[J]. 电子学报, 2021, 49(1): 183-191. |
[5] | 马少雄, 邱实, 唐颖, 张晓. 基于工地场景的深度学习目标跟踪算法[J]. 电子学报, 2020, 48(9): 1665-1671. |
[6] | 田增山, 廉颖慧, 周牧, 李泽, 金悦. 基于Wi-Fi多维参数特征的无源目标跟踪技术[J]. 电子学报, 2020, 48(8): 1572-1579. |
[7] | 付利华, 赵宇, 孙晓威, 卢中山, 王丹, 杨寒雪. 基于孪生网络的快速视频目标分割[J]. 电子学报, 2020, 48(4): 625-630. |
[8] | 陈辉, 贺忠良, 邓东明, 李国财. 高斯混合多伯努利滤波器基于柯西施瓦兹散度的传感器控制方法[J]. 电子学报, 2020, 48(4): 706-716. |
[9] | 叶瑾, 许枫, 杨娟, 王佳维. 一种基于多传感器的复合量测IMM-EKF数据融合算法[J]. 电子学报, 2020, 48(12): 2326-2330. |
[10] | 王树亮, 毕大平, 阮怀林, 杜明洋, 潘继飞. 基于信息熵准则的认知雷达机动目标跟踪算法[J]. 电子学报, 2019, 47(6): 1277-1284. |
[11] | 陈辉, 贺忠良, 连峰, 黎慧波. 基于高斯混合多目标滤波器的传感器控制策略[J]. 电子学报, 2019, 47(3): 521-530. |
[12] | 李姝怡, 程婷. 基于量测转换IMM的多普勒雷达机动目标跟踪[J]. 电子学报, 2019, 47(3): 538-544. |
[13] | 张浩为, 谢军伟, 葛佳昂, 宗彬锋, 路文龙. 基于波形调度的机动目标跟踪算法[J]. 电子学报, 2019, 47(3): 560-567. |
[14] | 戴忠华, 周穗华, 张宏欣, 单珊. 初值自适应的磁性目标跟踪方法[J]. 电子学报, 2019, 47(12): 2457-2464. |
[15] | 查宇飞, 吴敏, 库涛, 陈兵, 张园强. 基于位置敏感模型的深度跟踪算法研究[J]. 电子学报, 2019, 47(10): 2076-2082. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||