基于前景优化的视觉目标跟踪算法

doi:10.12263/DZXB.20210641

电子学报 ›› 2022, Vol. 50 ›› Issue (7): 1558-1566.DOI: 10.12263/DZXB.20210641

基于前景优化的视觉目标跟踪算法

谢青松¹, 刘晓庆², 安志勇¹, 李博¹

^1.山东工商学院计算机科学与技术学院，山东烟台 264000
^2.山东工商学院信息与电子工程学院，山东烟台 264000

收稿日期:2021-05-19 修回日期:2021-10-07 出版日期:2022-07-25
- 作者简介:
- 谢青松男，1965年1月生于山东青岛，现为山东工商学院计算机科学与技术学院教授、硕士生导师，主要研究方向为智能算法、图像识别、目标跟踪.E-mail: qs_xie@163.com
  刘晓庆女，1996年5月生于山东青岛，现为山东工商学院信息与电子工程学院硕士研究生，主要研究方向为目标跟踪.E-mail: 1322699199@qq.com
  安志勇（通讯作者）男，1975年10月生于山西怀仁，工学博士，现为山东工商学院计算机科学与技术学院副教授，研究方向为计算机视觉、目标跟踪等.E-mail:azytyut@163.com
  李博男，1980年3月生于辽宁锦州，东北大学信息科学与工程学院计算机系统结构博士，研究领域为大数据与机器学习.
- 基金资助:
- 国家自然科学基金 (62072285); 山东省自然科学基金 (ZR202102230438); 山东省重点研发计划 (软科学） (2020RKB01017); 山东工商学院校级教学改革项目 (11688202023)

Visual Object Tracking Algorithm Based on Foreground Optimization

XIE Qing-song¹, LIU Xiao-qing², AN Zhi-yong¹, LI Bo¹

^1.School of Computer Science and Technology, Shandong Technology and Business University, Yantai, Shandong 264000, China
^2.School of Information and Electronic Engineering, Shandong Technology and Business University, Yantai, Shandong 264000, China

Received:2021-05-19 Revised:2021-10-07 Online:2022-07-25 Published:2022-07-30
- Supported by:
- National Natural Science Foundation of China (62072285); Natural Science Foundation of Shandong Province, China (ZR202102230438); Key Research and Development Project of Shandong Province (Soft Science) (2020RKB01017); Teaching Reform Project of Shandong Technology and Business University (11688202023)

摘要/Abstract

摘要：

将目标分割技术引入跟踪领域是当前的研究热点.目前，基于分割的跟踪算法往往根据分割结果计算最小外接矩形，以此作为跟踪框，但复杂的目标运动使得跟踪框内包含较多背景，从而导致精度下降.针对该问题，本文提出了一种基于前景优化的视觉目标跟踪算法，将跟踪框的尺度和角度优化统一于前景优化框架中.首先评估跟踪框内的前景比例，若小于设定阈值，则对跟踪框分别进行尺度和角度优化；在尺度优化模块中，结合回归框计算跟踪框的条件概率，根据条件概率的结果分情形进行尺度优化；角度优化模块中，针对跟踪框设定多个偏移角度，利用前景IoU（Intersection over Union）极大策略选择最优跟踪框角度.结果证明，将本文方法应用于SiamMask算法，精度在VOT2016，VOT2018和VOT2019数据集分别提升约3.2%，3.7%和3.6%，而EAO分别提升约1.8%，1.9%和1.6%.另外，本文的方法针对基于分割的跟踪算法具有一定的普适性.

关键词: 目标分割, 目标跟踪, 前景优化, 尺度优化, 角度优化

Abstract:

The introduction of object segmentation technology into the tracking field is a current research hotspot. At present, the tracking algorithm based on segmentation often calculates the minimum bounding rectangle as the bounding box according to the segmentation result. However, the complex target movement makes the bounding box contain more background, which leads to a decrease in accuracy. In response to the problem, this paper proposes a visual object tracking algorithm based on foreground optimization, which unifies the optimization of the scale and angle in the bounding box into the foreground optimization frame. First, the foreground ratio in the bounding box is evaluated. If it is less than the set threshold, the scale and angle of the bounding box are optimized; in the scale optimization module, the conditional probability of the bounding box is calculated in combination with the regression box, and the scale is optimized according to the results of the conditional probability; in the angle optimization module, many deviation angles are set for the bounding box, and the optimal bounding box angle is chosen by the foreground IoU (Intersection over Union) maximum strategy. The proposed method is applied to the SiamMask algorithm. Results show that the accuracy is improved by about 3.2%, 3.7% and 3.6% in the VOT2016, VOT2018 and VOT2019 data sets, respectively, while EAO is increased by about 1.8%,1.9% and 1.6%, respectively. Moreover, our method has a certain universality for segmentation-based tracking algorithms.

Key words: object segmentation, object tracking, foreground optimization, scale optimization, angle optimization

中图分类号:

TP391.4

谢青松, 刘晓庆, 安志勇, 李博. 基于前景优化的视觉目标跟踪算法[J]. 电子学报, 2022, 50(7): 1558-1566.

XIE Qing-song, LIU Xiao-qing, AN Zhi-yong, LI Bo. Visual Object Tracking Algorithm Based on Foreground Optimization[J]. Acta Electronica Sinica, 2022, 50(7): 1558-1566.

图/表 5

参考文献 44

1	MATHURG, SOMWANSHID, BUNDELEM. Intelligent video surveillance based on object tracking[C]//Proceedings of the 2018 3rd International Conference and Workshops on Recent Advances and Innovations in Engineering. Jaipur, India: IEEE Press, 2018: 1-6.
2	GAJJARV, KHANDHEDIYAY, GURNANIA. Human detection and tracking for video surveillance: A cognitive science approach[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops. Venice, Italy: IEEE Press, 2017: 2805-2809.
3	CHENB, SAHDEVR, TSOTSOSJ, et al. Person following robot using selected online ada-boosting with stereo camera[C]//Proceedings of the 2017 14th Conference on Computer and Robot Vision. Edmonton, Canada: IEEE Press, 2017: 48-55.
4	张爱梅. 目标跟踪瞄准系统的虚拟现实研究[D].西安: 西安电子科技大学, 2004.
	ZHANGAi-mei. Virtual Reality Research of Target Tracking and Aiming System[D]. Xi'an: Xidian University, 2004. (in Chinese)
5	AGARWALN, CHIANGC, SHARMAA. A study on computer vision techniques for self-driving cars[C]//Proceedings of the International Conference on Frontier Computing. Singapore: Springer, 2019: 629-634.
6	孟琭, 杨旭. 目标跟踪算法综述[J]. 自动化学报, 2019, 45(7): 1244-1260.
	MENGLu, YANGXu. A survey of object tracking algorithms[J]. Acta Automatica Sinica, 2019, 45(7): 1244-1260. (in Chinese)
7	WUY, LIM J, YANGM H. Online object tracking: A benchmark[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Portland, USA: IEEE Press, 2013: 2411- 2418.
8	李玺, 查宇飞, 张天柱, 等. 深度学习的目标跟踪算法综述[J]. 中国图象图形学报, 2019, 24(12): 2057-2080.
	LIXi, ZHAYu-fei, ZHANGTian-zhu, et al. Survey of visual object tracking algorithms based on deep learning[J]. Journal of Image and Graphics, 2019, 24(12): 2057-2080. (in Chinese)
9	SUND Q, ROTHS, BLACKM J. Secrets of optical flow estimation and their principles[C]//Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, CA: IEEE, 2010: 2432-2439.
10	XUL, JIAJ Y, MATSUSHITAY. Motion detail preserving optical flow estimation[C]//Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE, 2010: 1293-1300.
11	NUMMIAROK, KOLLER-MEIERE, VAN GOOLL. An adaptive color-based particle filter[J]. Image and Vision Computing, 2003, 21(1): 99-110.
12	DUK, JUY F, JINY L, et al. Object tracking based on improved MeanShift and SIFT[C]//Proceedings of the 2nd International Conference on Consumer Electronics, Communications and Networks. Yichang, China: IEEE, 2012: 2716-2719.
13	EXNERD, BRUNSE, KURZD, et al. Fast and robust CAMShift tracking[C]//Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE, 2010: 9-16.
14	BOLMED, BEVERIDGEJ, DRAPERB, LUI Y. Visual object tracking using adaptive correlation filters[C]//Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE, 2010: 2544-2550.
15	DANELLJANM, et al. Discriminative scale space tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(8): 1561-1575.
16	丁新尧, 张鑫. 基于显著性特征的选择性目标跟踪算法[J]. 电子学报, 2020, 48(1): 118-123.
	DINGXin-yao, ZHANGXin. Visual tracking with salient features and selective mechanism[J]. Acta Electronica Sinica, 2020, 48(1): 118-123. (in Chinese)
17	王治丹, 蒋建国, 齐美彬, 等. 基于最大池图匹配的形变目标跟踪方法[J]. 电子学报, 2017, 45(3): 704-711.
	WANGZhi-dan, JIANGJian-guo, QIMei-bin, et al. Deformable object tracking based on max-pooling graph matching[J]. Acta Electronica Sinica, 2017, 45(3): 704-711. (in Chinese)
18	WANGN Y, YEUNGD Y. Learning a deep compact image representation for visual tracking[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada: Curran Associates Inc, 2013: 809-817.
19	ROUTL, MISHRAD, et al. Rotation adaptive visual object tracking with motion consistency[C]//Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision(WACV). Lake Tahoe, USA: IEEE, 2018: 1047-1055.
20	HUAY, ALAHARIK, SCHMIDC. Online object tracking with proposal selection[C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 3092-3100.
21	WANGQ, ZHANGL, BERTINETTOL, et al. Fast online object tracking and segmentation: A unifying approach[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE Press, 2019: 1328-1338.
22	PINHEIROP O, COLLOBERTR, DOLLARP. Learning to segment object candidates[C]//Proceedings of the Neural Information Processing Systems. Montreal, Canada:ACM, 2015: 1990-1998.
23	PINHEIROP O, LINT, COLLOBERTR, et al. Learning to refine object segments[C]//Proceedings of the European Conference on Computer Vision. Amsterdam: Springer, 2016: 75-91.
24	KRISTANM, LEONARDISA, MATASJ, et al. The visual object tracking vot2016 challenge results[C]//Proceedings of the European Conference on Computer Vision. Amsterdam: Springer, 2016: 777-823.
25	KRISTANM, LEONARDISA, MATASJ, et al. The sixth visual object tracking vot2018 challenge results[C]//Proceedings of the European Conference on Computer Vision. Munich, Germany: Springer,2018: 3-53.
26	KRISTANM, LEONARDISA, MATASJ, et al. The seventh visual object tracking vot2019 challenge results[C]//Proceedings of the IEEE International Conference on Computer Vision Workshop. Seoul, Korea: IEEE, 2019: 2206-2241.
27	BERTINETTOL, VALMADREJ, HENRIQUESJ F, et al. Fully-convolutional Siamese networks for object tracking[C]//Proceedings of the European Conference on Computer Vision. Amsterdam: Springer,2016: 850-865.
28	LIB, YANJ, WUW, et al. High performance visual tracking with Siamese region proposal network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake, USA: IEEE Press, 2018: 8971-8980.
29	LIB, WUW, WANGQ, et al. SiamRPN++: Evolution of Siamese visual tracking with very deep networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE Press, 2018: 4277-4286.
30	GUOD, WANGJ, CUIY, et al. SiamCAR: Siamese fully convolutional classification and regression for visual tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 6268-6276.
31	CHENZ, ZHONGB, LIG, et al. Siamese box adaptive network for visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 6667-6676.
32	HEA, LUOC, TIANX, et al. A twofold Siamese network for real-time object tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018:4834-4843.
33	DUY, LIUP, ZHAOW, et al. Correlation-guided attention for corner detection based visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 6835-6844.
34	RENS, HEK, GIRSHICKR, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137- 1149.
35	CHENB, TSOTSOS JOHNK. Fast visual object tracking using ellipse fitting for rotated bounding boxes[C]//2019 IEEE/CVF International Conference on Computer Vision Workshop(ICCVW). Seoul: IEEE, 2019: 2281-2289.
36	ZHUZ, WANGQ, LIB, et al. Distractor-aware Siamese networks for visual object tracking[C]//Proceedings of the IEEE European Conference on Computer Vision. Munich, Germany: Springer, 2018:103-119.
37	WANGG, LUOC, XIONGZ, et al. SPM-tracker: Series-parallel matching for real-time visual object tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019: 3638-3647.
38	VOIGTLAENDERP, LUITENETJ, et al. Siam R-CNN: Visual tracking by re-detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 6577-6587.
39	ZHANGZ, PENGH. Deeper and wider Siamese networks for real-time visual tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019: 4586-4595.
40	LIX, MAC, WUB, et al. Target-aware deep tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019:1369-1378.
41	DAIK, WANGD, LUH, et al. Visual tracking via adaptive spatially- regularized correlation filters[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019:4665-4674.
42	PARKE, BERGA C. Meta-tracker: Fast and robust online adaptation for visual object trackers[C]//Proceedings of the IEEE European Conference on Computer Vision. Munich, Germany: Springer, Cham, 2018:587-604.
43	ZHANGL, GONZALEZ-GARCIAA, WEIJERJ, et al. Learning the model update for Siamese trackers[C]//Proceedings of the IEEE International Conference on Computer Vision. Seoul, Korea: IEEE, 2019:4010-4019.
44	YANGT, XUP, HUR, et al. ROAM: Recurrently optimizing tracking model[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 6717-6726.

	VOT2016			VOT2018			VOT2019			平均速度(fps)
	A↑	R↓	EAO↑	A↑	R↓	EAO↑	A↑	R↓	EAO↑	平均速度(fps)
SiamMask	0.620	0.214	0.436	0.597	0.248	0.406	0.596	0.467	0.283	88
SiamMask_E	0.645	0.210	0.452	0.627	0.248	0.427	0.625	0.482	0.298	—
SiamMask+SO(MS)	0.644	0.228	0.437	0.623	0.267	0.410	0.622	0.487	0.294	88
SiamMask+SO(AS)	0.654	0.233	0.439	0.634	0.276	0.417	0.633	0.502	0.298	88
SiamMask+SO(MS)+AO	0.642	0.219	0.443	0.622	0.258	0.415	0.619	0.482	0.294	75
SiamMask+SO(AS)+AO	0.652	0.224	0.454	0.634	0.267	0.425	0.632	0.497	0.299	75

	VOT2016			VOT2018			VOT2019			平均速度(fps)
	A↑	R↓	EAO↑	A↑	R↓	EAO↑	A↑	R↓	EAO↑	平均速度(fps)
SiamMask	0.620	0.214	0.436	0.597	0.248	0.406	0.596	0.467	0.283	88
SiamMask_E	0.645	0.210	0.452	0.627	0.248	0.427	0.625	0.482	0.298	—
SiamMask+SO(MS)	0.644	0.228	0.437	0.623	0.267	0.410	0.622	0.487	0.294	88
SiamMask+SO(AS)	0.654	0.233	0.439	0.634	0.276	0.417	0.633	0.502	0.298	88
SiamMask+SO(MS)+AO	0.642	0.219	0.443	0.622	0.258	0.415	0.619	0.482	0.294	75
SiamMask+SO(AS)+AO	0.652	0.224	0.454	0.634	0.267	0.425	0.632	0.497	0.299	75

[1]	付利华, 赵宇, 姜涵煦, 赵茹, 吴会贤, 闫绍兴. 基于前景感知视觉注意的半监督视频目标分割[J]. 电子学报, 2022, 50(1): 195-206.
[2]	侯利明, 连峰, 谭顺成, 徐从安. 闪烁噪声统计特性未知情况下的鲁棒广义标签多伯努利滤波器[J]. 电子学报, 2021, 49(7): 1346-1353.
[3]	陈丹, 姚伯羽. 运动模型引导的自适应核相关目标跟踪方法[J]. 电子学报, 2021, 49(3): 550-558.
[4]	罗大鹏, 杜国庆, 曾志鹏, 魏龙生, 高常鑫, 陈应, 肖菲, 罗琛. 基于少量样本学习的多目标检测跟踪方法[J]. 电子学报, 2021, 49(1): 183-191.
[5]	马少雄, 邱实, 唐颖, 张晓. 基于工地场景的深度学习目标跟踪算法[J]. 电子学报, 2020, 48(9): 1665-1671.
[6]	田增山, 廉颖慧, 周牧, 李泽, 金悦. 基于Wi-Fi多维参数特征的无源目标跟踪技术[J]. 电子学报, 2020, 48(8): 1572-1579.
[7]	付利华, 赵宇, 孙晓威, 卢中山, 王丹, 杨寒雪. 基于孪生网络的快速视频目标分割[J]. 电子学报, 2020, 48(4): 625-630.
[8]	陈辉, 贺忠良, 邓东明, 李国财. 高斯混合多伯努利滤波器基于柯西施瓦兹散度的传感器控制方法[J]. 电子学报, 2020, 48(4): 706-716.
[9]	叶瑾, 许枫, 杨娟, 王佳维. 一种基于多传感器的复合量测IMM-EKF数据融合算法[J]. 电子学报, 2020, 48(12): 2326-2330.
[10]	王树亮, 毕大平, 阮怀林, 杜明洋, 潘继飞. 基于信息熵准则的认知雷达机动目标跟踪算法[J]. 电子学报, 2019, 47(6): 1277-1284.
[11]	陈辉, 贺忠良, 连峰, 黎慧波. 基于高斯混合多目标滤波器的传感器控制策略[J]. 电子学报, 2019, 47(3): 521-530.
[12]	李姝怡, 程婷. 基于量测转换IMM的多普勒雷达机动目标跟踪[J]. 电子学报, 2019, 47(3): 538-544.
[13]	张浩为, 谢军伟, 葛佳昂, 宗彬锋, 路文龙. 基于波形调度的机动目标跟踪算法[J]. 电子学报, 2019, 47(3): 560-567.
[14]	戴忠华, 周穗华, 张宏欣, 单珊. 初值自适应的磁性目标跟踪方法[J]. 电子学报, 2019, 47(12): 2457-2464.
[15]	查宇飞, 吴敏, 库涛, 陈兵, 张园强. 基于位置敏感模型的深度跟踪算法研究[J]. 电子学报, 2019, 47(10): 2076-2082.

基于前景优化的视觉目标跟踪算法

Visual Object Tracking Algorithm Based on Foreground Optimization

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献 44

相关文章 15

编辑推荐

Metrics