Cross-Scene Foreground Segmentation Algorithm Based on High-Level Feature Differencing Between Frames

ZHANG Jin; LI Yang; REN Chuan-lun; HUANG Lian; WANG Shuai-hui; DUAN Ye-xin; PAN Zhi-song; XIE Jun

doi:10.12263/DZXB.20200620

您当前的位置：

首页 >

文章列表页 >

Cross-Scene Foreground Segmentation Algorithm Based on High-Level Feature Differencing Between Frames

PAPERS | 更新时间：2025-12-08

- Cross-Scene Foreground Segmentation Algorithm Based on High-Level Feature Differencing Between Frames
- ACTA ELECTRONICA SINICA Vol. 49, Issue 10, Pages: 2032-2040(2021)
- 作者机构：
  
  1.陆军军事交通学院镇江校区，江苏镇江 212003
  2.陆军工程大学指挥控制工程学院，江苏南京 210007
  3.华北计算技术研究所，北京 100083
  4.海军装备部驻上海地区军事代表局，上海 200129
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20200620
  CLC： TP391.4;
- Received：28 June 2020，
  
  Revised：2020-12-03，
  
  Published：25 October 2021
- 稿件说明：
移动端阅览
张锦,李阳,任传伦等.基于帧间高级特征差分的跨场景视频前景分割算法[J].电子学报,2021,49(10):2032-2040.

ZHANG Jin,LI Yang,REN Chuan-lun,et al.Cross-Scene Foreground Segmentation Algorithm Based on High-Level Feature Differencing Between Frames[J].ACTA ELECTRONICA SINICA,2021,49(10):2032-2040.
张锦,李阳,任传伦等.基于帧间高级特征差分的跨场景视频前景分割算法[J].电子学报,2021,49(10):2032-2040. DOI： 10.12263/DZXB.20200620.

ZHANG Jin,LI Yang,REN Chuan-lun,et al.Cross-Scene Foreground Segmentation Algorithm Based on High-Level Feature Differencing Between Frames[J].ACTA ELECTRONICA SINICA,2021,49(10):2032-2040. DOI： 10.12263/DZXB.20200620.

摘要

当前基于深度学习的有监督前景分割方法得益于大量待分割场景的标注信息，其性能大幅超越传统的无监督方法.然而，获取高精度的像素级标注需要耗费大量的人力和时间成本，这严重限制了有监督算法在无标注场景的部署应用.为解决对场景监督信息依赖的问题，设计了一种与传统的帧间差分法相融合的跨场景深度学习架构，即帧间高级特征差分算法.该算法重点围绕时域变化等跨场景共性知识的迁移，在不依赖待分割场景监督信息的前提下实现高精度分割.面向五类不同模式的困难场景开展实验，本文算法的平均F值达到0.8719，超越了当前最高性能的有监督算法FgSegNet_v2（相同的跨场景条件下）和最佳的无监督算法SemanticBS.本文算法对QVGA视频（320×240）的处理速度达到35帧/s，具有较好的实时性.

Abstract

Benefiting from large amounts of ground-truths of to-be-segmented scenarios

deep-learning based and supervised foreground segmentation algorithms generally outperform conventional unsupervised methods. However

pixel-wise annotation is a tedious task

especially when it comes to the annotation of foreground moving objects. This seriously limits the deployment of a supervised algorithm in a wide range of scenes without ground-truths. To address the dependence on supervised information of to-be-segmented unseen scenes

we design an inter-frame high-level feature differencing algorithm with a deep learning architecture via integrating the traditional frame differencing method. The proposed algorithm leverages the transfer of cross-scene common knowledge

such as temporal changes

so as to achieve high performance for the scene in the absence of supervised information of to-be-segmented scenes. We evaluate our method on five challenging scenes with different patterns. The average F-Measure of our algorithm is 0.8719

which surpasses the current highest-performance (supervised) algorithm (FgSegNet_v2) under the cross-scene learning condition and the best unsupervised algorithm SemanticBS. Our method which can process a QVGA (320 × 240) video at 35 frames per second shows favorable real-time performance.

关键词

Keywords

references

LeCun Y , Bottou L , Bengio Y , et al . Gradient-based learning applied to document recognition [A]. Proceedings of the IEEE [C]. USA : IEEE , 1998 . 2278 － 2324 .

Fu J , Liu J , Tian H J , et al . Dual attention network for scene segmentation [A]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) [C]. Long Beach, CA, USA : IEEE , 2019 . 3141 － 3149 .

Sakkos D , Liu H , Han J G , et al . End-to-end video background subtraction with 3d convolutional neural networks [J]. Multimedia Tools and Applications , 2018 , 77 ( 17 ): 23023 － 23041 .

Wang Y , Luo Z M , Jodoin P M . Interactive deep learning method for segmenting moving objects [J]. Pattern Recognition Letters , 2017 , 96 : 66 － 75 .

Lim L A , Yalim Keles H . Foreground segmentation using convolutional neural networks for multiscale feature encoding [J]. Pattern Recognition Letters , 2018 , 112 : 256 － 262 .

Lim L A , Keles H Y . Learning multi-scale features for foreground segmentation [J]. Pattern Analysis and Applications , 2020 , 23 ( 3 ): 1369 － 1380 .

Babaee M , Dinh D T , Rigoll G . A deep convolutional neural network for video sequence background subtraction [J]. Pattern Recognition , 2018 , 76 : 635 － 649 .

Wang Y , Jodoin P M , Porikli F , et al . CDnet 2014: An expanded change detection benchmark dataset [A]. 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops [C]. Columbus, OH, USA : IEEE , 2014 . 393 － 400 .

Maddalena L , Petrosino A . Towards benchmarking scene background initialization [A]. New Trends in Image Analysis and Processing－ICIAP 2015 Workshops [C]. Cham, GER : Springer , 2015 . 469 － 476 . DOI: 10.1007/978－3－319－23222－5_57 http://dx.doi.org/10.1007/978－3－319－23222－5_57 .

Mandal M , Dhar V , Mishra A , et al . 3DFR: A swift 3D feature reductionist framework for scene independent change detection [J]. IEEE Signal Processing Letters , 2019 , 26 ( 12 ): 1882 － 1886 .

Bouwmans T , Javed S , Sultana M , et al . Deep neural network concepts for background subtraction: A systematic review and comparative evaluation [J]. Neural Networks , 2019 , 117 : 8 － 66 .

Braham M , Van Droogenbroeck M . Deep background subtraction with scene-specific convolutional neural networks [A]. 2016 International Conference on Systems, Signals and Image Processing (IWSSIP)[C] . Bratislava , Slovakia : IEEE , 2016 . 1 － 4 .

Chen M L , Wei X , Yang Q X , et al . Spatiotemporal GMM for background subtraction with superpixel hierarchy [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2018 , 40 ( 6 ): 1518 － 1525 .

Shi G M , Huang T , Dong W S , et al . Robust foreground estimation via structured Gaussian scale mixture modeling [J]. IEEE Transactions on Image Processing , 2018 , 27 ( 10 ): 4810 － 4824 .

常侃 , 张智勇 , 陈诚 , 等 . 采用低秩与加权稀疏分解的视频前景检测算法 [J]. 电子学报 , 2017 , 45 ( 9 ): 2272 － 2280 .

Chang K , Zhang Z Y , Chen C , et al . Video foreground detection by low-rank and reweighted sparse decomposition [J]. Acta Electronica Sinica , 2017 , 45 ( 9 ): 2272 － 2280 . (in Chinese)

秦晓燕 , 袁广林 , 李从利 , 等 . 一种快速鲁棒的视频序列运动目标检测方法 [J]. 电子学报 , 2017 , 45 ( 10 ): 2355 － 2361 .

Qin X Y , Yuan G L , Li C L , et al . An approach to fast and robust detecting of moving target in video sequences [J]. Acta Electronica Sinica , 2017 , 45 ( 10 ): 2355 － 2361 . (in Chinese)

Paul N , Singh A , Midya A , et al . Moving object detection using modified temporal differencing and local fuzzy thresholding [J]. The Journal of Supercomputing , 2017 , 73 ( 3 ): 1120 － 1139 .

Bilodeau G A , Jodoin J P , Saunier N . Change detection in feature space using local binary similarity patterns [A]. 2013 International Conference on Computer and Robot Vision [C]. Regina, SK, Canada : IEEE , 2013 . 106 － 112 .

St-Charles P L , Bilodeau G A , Bergevin R . A self-adjusting approach to change detection based on background word consensus [A]. 2015 IEEE Winter Conference on Applications of Computer Vision [C]. Waikoloa, HI, USA : IEEE , 2015 . 990 － 997 .

St-Charles P L , Bilodeau G A , Bergevin R . SuBSENSE: A universal change detection method with local adaptive sensitivity [J]. IEEE Transactions on Image Processing , 2015 , 24 ( 1 ): 359 － 373 .

Braham M , Piérard S , Van Droogenbroeck M . Semantic background subtraction [A]. 2017 IEEE International Conference on Image Processing (ICIP) [C]. Beijing, China : IEEE , 2017 . 4552 － 4556 .

Sultana M , Mahmood A , Javed S , et al . Unsupervised deep context prediction for background estimation and foreground segmentation [J]. Machine Vision and Applications , 2019 , 30 ( 3 ): 375 － 395 .

Zeng D D , Zhu M , Kuijper A . Combining background subtraction algorithms with convolutional neural network [J]. Journal of Electronic Imaging , 2019 , 28 ( 1 ): 013011 .

Zhang J , Li Y , Chen F Q , et al . X-net: A binocular summation network for foreground segmentation [J]. IEEE Access , 2019 , 7 : 71412 － 71422 .

Chen L C , Zhu Y K , Papandreou G , et al . Encoder-decoder with atrous separable convolution for semantic image segmentation [A]. European Conference on Computer Vision [C]. Cham, GER : Springer , 2018 . 833 － 851 . DOI: 10.1007/978－3－030－01234－2_49 http://dx.doi.org/10.1007/978－3－030－01234－2_49 .

Yang M K , Yu K , Zhang C , et al . DenseASPP for semantic segmentation in street scenes [A]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition [C]. Salt Lake City, UT, USA : IEEE , 2018 . 3684 － 3692 .

Lin T Y , Goyal P , Girshick R , et al . Focal loss for dense object detection [A]. 2017 IEEE International Conference on Computer Vision (ICCV) [C]. Venice, Italy : IEEE , 2017 . 2999 － 3007 .

Bianco S , Ciocca G , Schettini R . Combination of video change detection algorithms by genetic programming [J]. IEEE Transactions on Evolutionary Computation , 2017 , 21 ( 6 ): 914 － 928 .

Chen M L , Yang Q X , Li Q , et al . Spatiotemporal background subtraction using minimum spanning tree and optical flow [A]. European Conference on Computer Vision [C]. Cham, GER : Springer , 2014 . 521 － 534 . DOI: 10.1007/978－3－319－10584－0_34 http://dx.doi.org/10.1007/978－3－319－10584－0_34 .

Sajid H , Cheung S C S . Universal multimode background subtraction [J]. IEEE Transactions on Image Processing , 2017 , 26 ( 7 ): 3249 － 3260 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Locomotion Intention Prediction via Multi-Sensor Fusion and Transfer Learning for Lower Limb Exoskeletons

Hybrid Task Assignment for Crowd Sensing Based on Mobility Prediction

Semantic Comparison Model for Binary Programs Based on Statistical Reasoning

Zero Sample Fault Diagnosis Based on Transfer Learning

Radar Target Recognition Method for HRRP Based on Geodesic Flow Kernel

Related Author

YU Zhi-peng

WANG Mei-ling

WANG Cheng-jun

LING Liu-yi

JIN Li

JIANG Wei-jin

NIE Cai-yan

LIU Qian

Related Institution

Institute of Intelligent Machinery, Hefei Institutes of Physical Science, Chinese Academy of Sciences

Anhui Key Laboratory of Bionic Sensing and Advanced Robot technology

State Key Laboratory of Digital Intelligent Technology for Unmanned Coal Mining, Anhui University of Science Technology

School of Artificial Intelligence, Anhui University of Science Technology

Hunan University of Information Technology

⁰