基于深度帧差卷积神经网络的运动目标检测方法研究

doi:10.3969/j.issn.0372-2112.2020.12.014

电子学报 ›› 2020, Vol. 48 ›› Issue (12): 2384-2393.DOI: 10.3969/j.issn.0372-2112.2020.12.014

基于深度帧差卷积神经网络的运动目标检测方法研究

欧先锋¹, 晏鹏程¹, 王汉谱¹, 涂兵¹, 何伟¹, 张国云¹, 徐智²

1. 湖南理工学院信息科学与工程学院机器视觉与人工智能研究中心, 湖南岳阳 414006;
2. 桂林电子科技大学广西图像图形智能处理重点实验室, 广西桂林 541004

收稿日期:2020-04-23 修回日期:2020-09-27 出版日期:2020-12-25
- 通讯作者:
- 张国云, 徐智
- 作者简介:
- 欧先锋 男,1983年7月生于湖南郴州.现为湖南理工学院副教授、硕士生导师.主要研究方向为计算机视觉、高光谱遥感图像处理.E-mail:ouxf@hnist.edu.cn;晏鹏程 男,1995年6月生于湖南益阳.现为湖南理工学院硕士生.主要研究方向为深度学习框架与算法、计算机视觉.E-mail:530865028@qq.com;王汉谱 男,1997年4月生于江苏盐城.现为湖南理工学院硕士生.主要研究方向为深度学习框架与算法、计算机视觉.E-mail:1215051195@qq.com;涂兵男,1983年1月生于湖南岳阳.现为湖南理工学院副教授、硕士生导师.主要研究方向为计算机视觉、高光谱遥感图像处理.E-mail:tubing@hnist.edu.cn;何伟男,1983年1月生于湖南岳阳.现为湖南理工学院副教授、硕士生导师.主要研究方向为计算机视觉、机器学习.E-mail:hewei@hnist.edu.cn
- 基金资助:
- 湖南省自然科学基金项目 (No.2020JJ4340，No.2020JJ4343）; 国家自然科学基金 (No.61662014）; 湖南省教育厅优秀青年项目 (No.19B245）; 湖南省研究生教育创新工程和专业能力提升工程项目 (No.CX20201114）; 湖南省三维重建与智能应用技术工程研究中心 (No.2019-430602-73-03-006049）; 湖南省应急通信工程技术研究中心 (No.2018TP2022）; 广西科技基地和人才专项 (No.AD19110022）

Research of Moving Object Detection Based on Deep Frame Difference Convolution Neural Network

OU Xian-feng¹, YAN Peng-cheng¹, WANG Han-pu¹, TU Bing¹, HE Wei¹, ZHANG Guo-yun¹, XU Zhi²

1. School of Information and Communication Engineering, Machine Vision & Artificial Intelligence Research Center, Hunan Institute of Science and Technology, Yueyang, Hunan 414006, China;
2. Guangxi Key Laboratory of Images and Graphics Intelligent Processing, Guilin University of Electronics Technology, Guilin, Guangxi 541004, China

Received:2020-04-23 Revised:2020-09-27 Online:2020-12-25 Published:2020-12-25
- Corresponding author:
- ZHANG Guo-yun, XU Zhi
- Supported by:
- Program of Natural Science Foundation of Hunan Province, China (No.2020JJ4340, No.2020JJ4343); National Natural Science Foundation of China (No.61662014); Excellent Youth Program of Education Department of Hunan Province (No.19B245); Hunan Postgraduate Education Innovation Project and Professional Ability Enhancement Project (No.CX20201114); Engineering Research Center of 3D Reconstruction and Intelligent Application Technology of Hunan Province (No.2019-430602-73-03-006049); Emergency Communication Engineering Technology Research Center of Hunan Province (No.2018TP2022); Guangxi Science and Technology Base and Talent Program (No.AD19110022)

摘要/Abstract

摘要： 复杂场景中的运动目标检测是计算机视觉领域的重要问题，其检测准确度仍然是一大挑战.本文提出并设计了一种用于复杂场景中运动目标检测的深度帧差卷积神经网络（Deep Difference Convolutional Neural Network，DFDCNN）.DFDCNN由DifferenceNet和AppearanceNet组成，不需要后处理就可以预测分割前景像素.DifferenceNet具有孪生Encoder-Decoder结构，用于学习两个连续帧之间的变化，从输入（t帧和t+1帧）中获取时序信息；AppearanceNet用于从输入（t帧）中提取空间信息，并与时序信息融合；同时，通过多尺度特征图融合和逐步上采样来保留多尺度空间信息，以提高网络对小目标的敏感性.在公开标准数据集CDnet2014和I2R上的实验结果表明：DFDCNN不仅在动态背景、光照变化和阴影存在的复杂场景中具有更好的检测性能，而且在小目标存在的场景中也具有较好的检测效果.

关键词: 运动目标检测, 复杂场景, 深度帧差卷积神经网络, 时序信息, 空间信息, 多尺度特征图融合

Abstract: Moving object detection in complex scenes is an important problem in computer vision domain, and the detection accuracy is still a great challenge. In this paper, we propose and design a deep frame difference convolution neural network (DFDCNN) for moving object detection in complex scenes. DFDCNN consists of DifferenceNet and AppearanceNet, which can predict and segment the foreground pixels simultaneously without post-processing. DifferenceNet has Siamese Encoder-Decoder structure, which is used to learn changes between two consecutive frames and to obtain temporal information from inputs, while AppearanceNet is used to extract spatial information from the input frame, and fuse the temporal information and spatial information by fusion of feature maps. Finally, multi-scale spatial information is retained through multi-scale feature map fusion and stepwise up-sampling to improve the sensitivity to small objects. Experiments on two public standard datasets: CDnet2014 and I2R demonstrate that the proposed DFDCNN outperforms the classic algorithms significantly from both qualitative and quantitative aspects. The experimental results illustrate that the proposed DFDCNN shows much better detection performance in complex scenes where dynamic background, illumination variation and shadow exist, and there is improvement for scenes, in which small objects exist.

Key words: moving object detection, complex scenes, deep frame difference convolutional neural network, temporal information, spatial information, multi-scale feature map fusion

中图分类号:

TP183

欧先锋, 晏鹏程, 王汉谱, 等. 基于深度帧差卷积神经网络的运动目标检测方法研究[J]. 电子学报, 2020, 48(12): 2384-2393.

OU Xian-feng, YAN Peng-cheng, WANG Han-pu, et al. Research of Moving Object Detection Based on Deep Frame Difference Convolution Neural Network[J]. Acta Electronica Sinica, 2020, 48(12): 2384-2393.

参考文献

[1] Yazdi M,Bouwmans T.New trends on moving object detection in video images captured by a moving camera:A survey[J].Computer Science Review,2018,28:157-177.
[2] Joshi K A,Thakore D G.A survey on moving object detection and tracking in video surveillance system[J].International Journal of Soft Computing and Engineering,2012,2(3):44-48.
[3] Mabrouk A B,Zagrouba E.Abnormal behavior recognition for intelligent video surveillance systems:A review[J].Expert Systems with Applications,2018,91:480-491.
[4] Wang Q,Zhang L,et al.Fast online object tracking and segmentation:A unifying approach[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].Long Beach,USA:ACM,2019.1328-1338.
[5] Zeng Z,Li Z,et al.Two-stream multirate recurrent neural network for video-based pedestrian reidentification[J].IEEE Transactions on Industrial Informatics,2017,14(7):3179-3186.
[6] Li L,Hu Q,et al.Moving object detection in video via hierarchical modeling and alternating optimization[J].IEEE Transactions on Image Processing,2018,28(4):2021-2036.
[7] Patil P W,Murala S.Msfgnet:A novel compact end-to-end deep network for moving object detection[J].IEEE Transactions on Intelligent Transportation Systems,2018,20(11):4066-4077.
[8] Lim L A,Keles H Y.Foreground segmentation using convolutional neural networks for multiscale feature encoding[J].Pattern Recognition Letters,2018,112:256-262.
[9] Wei Z,Li P,et al.A foreground-background segmentation algorithm for video sequences[A].Proceedings of the International Symposium on Distributed Computing and Applications for Business,Engineering and Science[C].Washington DC,USA:ACM,2015.340-343.
[10] Xu Z,Zhang D,et al.Moving object detection based on improved three frame difference and background subtraction[A].Proceedings of the International Conference on Industrial Informatics-Computing Technology,Intelligent Technology,Industrial Information Integration[C].Wuhan,China:ACM,2017.79-82.
[11] Xiang C,Zhang L,et al.MS-CapsNet:A novel multi-scale capsule network[J].IEEE Signal Processing Letters,2018,25(12):1850-1854.
[12] Han C,Duan Y,et al.Dense convolutional networks for semantic segmentation[J].IEEE Access,2019,7:43369-43382.
[13] Ou X,Yan P,et al.Moving object detection method via ResNet-18 with encoder-decoder structure in complex scenes[J].IEEE Access,2019,7:108152-108160.
[14] Lim K,Jang W D,et al.Background subtraction using encoder-decoder structured convolutional neural network[A].Proceedings of the IEEE International Conference on Advanced Video and Signal Based Surveillance[C].Lecce,Italy:ACM,2017.1-6.
[15] Dosovitskiy A,Fischer P,et al.Flownet:Learning optical flow with convolutional networks[A].Proceedings of the IEEE International Conference on Computer Vision[C].Washington DC,USA:ACM,2015.2758-2766.
[16] Chopra S,Hadsell R,et al.Learning a similarity metric discriminatively,with application to face verification[A].IEEE Computer Society Conference on Computer Vision and Pattern Recognition[C].San Diego,USA:ACM,2005.539-546.
[17] Simonyan K,Zisserman A.Very Deep Convolutional Networks for Large-scale Image Recognition[DB/OL].arXiv preprint arXiv:1409.1556,2014.
[18] He K,Zhang X,et al.Deep residual learning for image recognition[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].Las Vegas,USA:ACM,2016.770-778.
[19] Huang G,Liu Z,et al.Densely connected convolutional networks[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].Honolulu,USA:ACM,2017.4700-4708.
[20] Long J,Shelhamer E,et al.Fully convolutional networks for semantic segmentation[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].Boston,USA:ACM,2015.3431-3440.
[21] Ioffe S,Szegedy C.Batch Normalization:Accelerating Deep Network Training by Reducing Internal Covariate Shift[DB/OL].arXiv preprint arXiv:1502.03167,2015.
[22] Nair V,Hinton G E.Rectified linear units improve restricted boltzmann machines[A].Proceedings of the International Conference on Machine Learning[C].Washington DC,USA:ACM,2010.807-814.
[23] Li L,Huang W,et al.Statistical modeling of complex backgrounds for foreground object detection[J].IEEE Transactions on Image Processing,2004,13(11):1459-1472.
[24] Wang Y,Jodoin P M,et al.CDnet 2014:An expanded change detection benchmark dataset[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].Columbus,USA:ACM,2014.387-394.
[25] Maddalena L,Petrosino A.The 3dSOBS+ algorithm for moving object detection[J].Computer Vision and Image Understanding,2014,122:65-73.
[26] Liu X,Zhao G,et al.Background subtraction based on low-rank and structured sparse decomposition[J].IEEE Transactions on Image Processing,2015,24(8):2502-2514.
[27] Yong H,Meng D,et al.Robust online matrix factorization for dynamic background subtraction[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(7):1726-1740.
[28] Wang K,Gou C,et al.M4CD:A robust change detection method for intelligent visual surveillance[J].IEEE Access,2018,6:15505-15520.
[29] Bianco S,Ciocca G,et al.Combination of video change detection algorithms by genetic programming[J].IEEE Transactions on Evolutionary Computation,2017,21(6):914-928.
[30] Isik S,Özkan K,et al.SWCD:a sliding window and self-regulated learning-based background updating method for change detection in videos[J].Journal of Electronic Imaging,2018,27(2):023002.
[31] Lee S,Lee G,et al.Wisenetmd:Motion detection using dynamic background region analysis[J].Symmetry,2019,11(5):621.
[32] Babaee M,Dinh D T,et al.A deep convolutional neural network for video sequence background subtraction[J].Pattern Recognition,2018,76:635-649.
[33] Tezcan O,Ishwar P,et al.BSUV-Net:a fully-convolutional neural network for background subtraction of unseen videos[A].IEEE Winter Conference on Applications of Computer Vision[C].Snowmass,USA:ACM,2020.2774-2783.

基于深度帧差卷积神经网络的运动目标检测方法研究

Research of Moving Object Detection Based on Deep Frame Difference Convolution Neural Network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	王洪雁, 张海坤, 罗宇华, 汪祖民. 复杂动态背景下基于群稀疏的运动目标检测方法[J]. 电子学报, 2021, 49(12): 2330-2338.
[2]	张泰江, 李勇军, 赵尚弘, 郑永兴, 辛宁. 基于代数连通度优化的空间信息网络分布式拓扑控制方法[J]. 电子学报, 2020, 48(6): 1162-1168.
[3]	张绍泉, 黄志浩, 邓承志, 李璠, 徐晨光, 吴朝明, 汪胜前. 光谱加权协同稀疏和全变差正则化高光谱图像解混[J]. 电子学报, 2020, 48(12): 2453-2461.
[4]	葛利跃, 张聪炫, 陈震, 黎明, 陈昊. 相互结构引导滤波TV-L1变分光流估计[J]. 电子学报, 2019, 47(3): 707-713.
[5]	陈荣元, 徐雪松, 申立智, 刘跃华, 陈浪. 模糊区域级MRF方法在城镇自动识别中的应用[J]. 电子学报, 2019, 47(2): 489-494.
[6]	王星宇, 李勇军, 赵尚弘, 曹桂兴, 谭庆贵, 郑永兴, 姜勇. 基于IEEE802.11的星间链路最短接入时延退避算法[J]. 电子学报, 2018, 46(12): 2936-2941.
[7]	常侃, 张智勇, 陈诚, 覃团发. 采用低秩与加权稀疏分解的视频前景检测算法[J]. 电子学报, 2017, 45(9): 2272-2280.
[8]	秦晓燕, 袁广林, 李从利, 张旭. 一种快速鲁棒的视频序列运动目标检测方法[J]. 电子学报, 2017, 45(10): 2355-2361.
[9]	于红绯, 刘威, 袁淮, 赵宏. 基于子块运动补偿的运动目标检测[J]. 电子学报, 2017, 45(1): 173-180.
[10]	刘龙, 樊波阳, 刘金星, 杨乐超. 面向运动目标检测的粒子滤波视觉注意力模型[J]. 电子学报, 2016, 44(9): 2235-2241.
[11]	毕福昆, 陈禾, 师皓, 章菲菲. 机载复杂遥感场景下特定建筑区检测跟踪算法[J]. 电子学报, 2016, 44(6): 1394-1399.
[12]	周伟, 孙玉宝, 刘青山, 吴敏. 运动目标检测的l₀群稀疏RPCA模型及其算法[J]. 电子学报, 2016, 44(3): 627-632.
[13]	杨志伟, 张蕊, 许华健, 廖桂生, 杨剑. 多特征辅助的SAR-GMTI雷达扩展动目标检测方法[J]. 电子学报, 2016, 44(12): 2916-2923.
[14]	蒋建国, 金玉龙, 齐美彬, 詹曙. 基于稀疏表达残差的自然场景运动目标检测[J]. 电子学报, 2015, 43(9): 1738-1744.
[15]	史洪印, 张诺. 基于稀疏表示和道路辅助的单幅SAR图像运动目标检测方法[J]. 电子学报, 2015, 43(3): 431-439.